[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:57:37 dist-test-slave-9zdv krb5kdc[3038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:57:37 dist-test-slave-9zdv krb5kdc[3038](info): set up 2 sockets
Jun 21 06:57:37 dist-test-slave-9zdv krb5kdc[3038](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260621 06:57:38.287380 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:57:38 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:57:38.456964 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35457
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:38.792089 3051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:38.792390 3051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:38.792471 3051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:38.799712 3051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:57:38.799821 3051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:38.799867 3051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:38.799938 3051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:57:38.799981 3051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:57:38.809406 3051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:38.810642 3051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:38.811889 3051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:38.823874 3059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:38.823839 3057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:38.823904 3056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:38.824194 3051 server_base.cc:1034] running on GCE node
I20260621 06:57:38.825235 3051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:38.826578 3051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:38.827786 3051 hybrid_clock.cc:648] HybridClock initialized: now 1782025058827719 us; error 53 us; skew 500 ppm
I20260621 06:57:38.828083 3051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:38 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025058, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:38.831234 3051 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:57:38.832731 3051 webserver.cc:460] Webserver started at http://127.2.245.254:37947/ using document root <none> and password file <none>
I20260621 06:57:38.833457 3051 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:38.833587 3051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:38.833900 3051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:38.837416 3051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "82f15323512246a29f593e59bf99db38"
format_stamp: "Formatted at 2026-06-21 06:57:38 on dist-test-slave-9zdv"
server_key: "adf37ede2987f2560ca8a2ba85f30e39"
server_key_iv: "1af4da0fa00294cde41cbba31e0b6b79"
server_key_version: "encryptionkey@0"
I20260621 06:57:38.838189 3051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "82f15323512246a29f593e59bf99db38"
format_stamp: "Formatted at 2026-06-21 06:57:38 on dist-test-slave-9zdv"
server_key: "adf37ede2987f2560ca8a2ba85f30e39"
server_key_iv: "1af4da0fa00294cde41cbba31e0b6b79"
server_key_version: "encryptionkey@0"
I20260621 06:57:38.843444 3051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:57:38.846782 3066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:38.848351 3051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:57:38.848541 3051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "82f15323512246a29f593e59bf99db38"
format_stamp: "Formatted at 2026-06-21 06:57:38 on dist-test-slave-9zdv"
server_key: "adf37ede2987f2560ca8a2ba85f30e39"
server_key_iv: "1af4da0fa00294cde41cbba31e0b6b79"
server_key_version: "encryptionkey@0"
I20260621 06:57:38.848788 3051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:38.862787 3051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:38.866801 3051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:38.867156 3051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:38.885839 3051 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35457
I20260621 06:57:38.885864 3117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35457 every 8 connection(s)
I20260621 06:57:38.887368 3051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:57:38.891177 3118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:38.892903 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3051
I20260621 06:57:38.893208 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:57:38.895364 3031 external_mini_cluster.cc:1428] Setting key 87d954f403add87c26828890afd92413
I20260621 06:57:38.902467 3118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38: Bootstrap starting.
I20260621 06:57:38.905448 3118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:38.906410 3118 log.cc:824] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:38.909415 3118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38: No bootstrap required, opened a new log
I20260621 06:57:38.918674 3118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } }
I20260621 06:57:38.919303 3118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:38.919430 3118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f15323512246a29f593e59bf99db38, State: Initialized, Role: FOLLOWER
I20260621 06:57:38.920173 3118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } }
I20260621 06:57:38.920477 3118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:57:38.920640 3118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:57:38.920822 3118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:38.923300 3118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } }
I20260621 06:57:38.923804 3118 leader_election.cc:302] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82f15323512246a29f593e59bf99db38; no voters:
I20260621 06:57:38.924424 3118 leader_election.cc:288] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:57:38.924623 3123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 1 FOLLOWER]: Leader election won for term 1
Jun 21 06:57:38 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:38.925279 3123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [term 1 LEADER]: Becoming Leader. State: Replica: 82f15323512246a29f593e59bf99db38, State: Running, Role: LEADER
I20260621 06:57:38.925884 3123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } }
I20260621 06:57:38.926959 3118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:57:38.928858 3124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f15323512246a29f593e59bf99db38. Latest consensus state: current_term: 1 leader_uuid: "82f15323512246a29f593e59bf99db38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } } }
I20260621 06:57:38.929324 3124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:38.931563 3125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82f15323512246a29f593e59bf99db38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f15323512246a29f593e59bf99db38" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35457 } } }
I20260621 06:57:38.931928 3125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:38.933382 3128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:57:38.941582 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:38.899372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.2.245.254:35457)
0621 06:57:38.900396 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:38.900417 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:38.901754 (+ 1337us) server_negotiation.cc:407] Connection header received
0621 06:57:38.911082 (+ 9328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:38.911110 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:38.911626 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:38.912094 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:38.914909 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:38.917088 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:38.919010 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:38.920068 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:38.920265 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:38.927092 (+ 6827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:38.927107 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:38.927132 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:38.927186 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:38.932720 (+ 5534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:38.933862 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:38.933870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:38.933877 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:38.934082 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:38.934715 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:38.934720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:38.934724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:38.935561 (+ 837us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:38.938794 (+ 3233us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:38.939183 (+ 389us) server_negotiation.cc:299] Negotiation successful
0621 06:57:38.939760 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":485,"threads_started":1}
I20260621 06:57:38.942221 3128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:57:38.956207 3128 catalog_manager.cc:1347] Generated new cluster ID: 120dfdeb9e754c0a9b63d4c9801ea7d7
I20260621 06:57:38.956362 3128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:57:38.983649 3128 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:57:38.985630 3128 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:57:38.994421 3128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82f15323512246a29f593e59bf99db38: Generated new TSK 0
I20260621 06:57:38.995326 3128 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:57:39.048226 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:39.393018 3144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:39.393313 3144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:39.393492 3144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:39.400820 3144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:39.400938 3144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:39.401135 3144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:57:39.410900 3144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:39.412262 3144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:39.413625 3144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:39.426720 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:39.426770 3150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:39.427609 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:39.427637 3144 server_base.cc:1034] running on GCE node
I20260621 06:57:39.428589 3144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:39.429559 3144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:39.430814 3144 hybrid_clock.cc:648] HybridClock initialized: now 1782025059430740 us; error 48 us; skew 500 ppm
I20260621 06:57:39.431156 3144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:39 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025059, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:39.434427 3144 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:57:39.435936 3144 webserver.cc:460] Webserver started at http://127.2.245.193:37927/ using document root <none> and password file <none>
I20260621 06:57:39.436543 3144 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:39.436656 3144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:39.436920 3144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:39.439749 3144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "887f61da11544bdea75eba279efaf0c1"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "d7f11b633bb920c3c4152199c0351d84"
server_key_iv: "9e774b94fe228d7504809a703dd12c72"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.440428 3144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "887f61da11544bdea75eba279efaf0c1"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "d7f11b633bb920c3c4152199c0351d84"
server_key_iv: "9e774b94fe228d7504809a703dd12c72"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.445370 3144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:57:39.448766 3159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:39.450167 3144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 06:57:39.450368 3144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "887f61da11544bdea75eba279efaf0c1"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "d7f11b633bb920c3c4152199c0351d84"
server_key_iv: "9e774b94fe228d7504809a703dd12c72"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.450575 3144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:39.483348 3144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:39.487370 3144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:39.487671 3144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:39.488652 3144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:39.490032 3144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:39.490118 3144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:39.490281 3144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:39.490347 3144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:39.523617 3144 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:35099
I20260621 06:57:39.523663 3271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:35099 every 8 connection(s)
I20260621 06:57:39.524955 3144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:57:39.527937 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3144
I20260621 06:57:39.528163 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:57:39.528971 3031 external_mini_cluster.cc:1428] Setting key fddb314911930ae9ee3f0bb3ea1f37ae
Jun 21 06:57:39 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025059, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:57:39.547137 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:39.527587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:53813 (local address 127.2.245.254:35457)
0621 06:57:39.528176 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:39.528184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:39.529403 (+ 1219us) server_negotiation.cc:407] Connection header received
0621 06:57:39.531723 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:39.531727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:39.531813 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:39.531907 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:39.534201 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:39.535125 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:39.536978 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:39.537364 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:39.537463 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:39.541114 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:39.541119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:39.541122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:39.541155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:39.543820 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:39.544591 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:39.544599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:39.544604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:39.544728 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:39.545203 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:39.545209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:39.545212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:39.545485 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:39.545635 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:39.546418 (+ 783us) server_negotiation.cc:299] Negotiation successful
0621 06:57:39.546680 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":283,"threads_started":1}
I20260621 06:57:39.548282 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:39.528292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35457 (local address 127.2.245.193:53813)
0621 06:57:39.529199 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:39.529252 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:39.531410 (+ 2158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:39.532208 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:39.532223 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:39.533101 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:39.533974 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:39.534001 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:39.535271 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:39.535279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:39.536824 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:39.536836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:39.537551 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:39.537555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:39.537737 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:39.538454 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:39.538480 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:39.540878 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:39.543981 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:39.543991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:39.544009 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:39.544464 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:39.544840 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:39.544845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:39.544854 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:39.545073 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:39.545621 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:39.545629 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:39.545984 (+ 355us) client_negotiation.cc:769] Sending connection context
0621 06:57:39.546329 (+ 345us) client_negotiation.cc:240] Negotiation successful
0621 06:57:39.546927 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":382,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:57:39.549950 3272 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35457
I20260621 06:57:39.550292 3272 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:39.551240 3272 heartbeater.cc:511] Master 127.2.245.254:35457 requested a full tablet report, sending...
I20260621 06:57:39.553860 3083 ts_manager.cc:194] Registered new tserver with Master: 887f61da11544bdea75eba279efaf0c1 (127.2.245.193:35099)
I20260621 06:57:39.556152 3083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:53813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:57:39.570852 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:39.887636 3278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:39.887964 3278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:39.888149 3278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:39.895403 3278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:39.895540 3278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:39.895714 3278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:57:39.913208 3278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:39.914897 3278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:39.916601 3278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:39.935387 3284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:39.935729 3286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:39.935897 3283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:39.936798 3278 server_base.cc:1034] running on GCE node
I20260621 06:57:39.937583 3278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:39.938840 3278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:39.940142 3278 hybrid_clock.cc:648] HybridClock initialized: now 1782025059940067 us; error 57 us; skew 500 ppm
I20260621 06:57:39.940563 3278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:39 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025059, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:39.944749 3278 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:57:39.946218 3278 webserver.cc:460] Webserver started at http://127.2.245.194:39803/ using document root <none> and password file <none>
I20260621 06:57:39.946987 3278 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:39.947140 3278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:39.947471 3278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:39.950495 3278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "203bfc17fd9c44fcbd68e942ee4f03f0"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "061a462820406a1506d8cd10a1c57944"
server_key_iv: "9b239afb58b4a366b60f955068f486b0"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.951709 3278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "203bfc17fd9c44fcbd68e942ee4f03f0"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "061a462820406a1506d8cd10a1c57944"
server_key_iv: "9b239afb58b4a366b60f955068f486b0"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.957285 3278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:57:39.960714 3293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:39.961995 3278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 06:57:39.962167 3278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "203bfc17fd9c44fcbd68e942ee4f03f0"
format_stamp: "Formatted at 2026-06-21 06:57:39 on dist-test-slave-9zdv"
server_key: "061a462820406a1506d8cd10a1c57944"
server_key_iv: "9b239afb58b4a366b60f955068f486b0"
server_key_version: "encryptionkey@0"
I20260621 06:57:39.962353 3278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:39.984433 3278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:39.990859 3278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:39.991313 3278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:39.992425 3278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:39.993734 3278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:39.993808 3278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:39.993954 3278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:39.994012 3278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:40.025242 3278 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:45859
I20260621 06:57:40.025327 3405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:45859 every 8 connection(s)
I20260621 06:57:40.026562 3278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:57:40.036228 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3278
I20260621 06:57:40.036429 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:57:40.036897 3031 external_mini_cluster.cc:1428] Setting key 2c306c020a6a403f2cf2e73a8bef536e
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025059, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:40.047619 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.029178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59941 (local address 127.2.245.254:35457)
0621 06:57:40.029496 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.029502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.030718 (+ 1216us) server_negotiation.cc:407] Connection header received
0621 06:57:40.032507 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.032511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.032580 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.032673 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.034355 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.035148 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.036812 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.037452 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.037584 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.041629 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.041634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.041637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.041669 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.044489 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.045216 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.045225 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.045231 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.045384 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.045762 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.045769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.045775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.046148 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.046293 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.046912 (+ 619us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.047201 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260621 06:57:40.048357 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.029826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35457 (local address 127.2.245.194:59941)
0621 06:57:40.030540 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:40.030577 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:40.032219 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:40.032884 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:40.032892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:40.033376 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:40.034019 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:40.034038 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.035295 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.035299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:40.036682 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:40.036694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.037702 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.037711 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:40.037945 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.038598 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:40.038622 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:40.041399 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:40.044617 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:40.044623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:40.044636 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:40.045098 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:40.045486 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:40.045489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:40.045491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:40.045656 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:40.046261 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:40.046267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:40.046563 (+ 296us) client_negotiation.cc:769] Sending connection context
0621 06:57:40.046778 (+ 215us) client_negotiation.cc:240] Negotiation successful
0621 06:57:40.047222 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":275,"threads_started":1}
I20260621 06:57:40.049870 3406 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35457
I20260621 06:57:40.050213 3406 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:40.051192 3406 heartbeater.cc:511] Master 127.2.245.254:35457 requested a full tablet report, sending...
I20260621 06:57:40.052832 3083 ts_manager.cc:194] Registered new tserver with Master: 203bfc17fd9c44fcbd68e942ee4f03f0 (127.2.245.194:45859)
I20260621 06:57:40.053789 3083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:59941
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:57:40.076359 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:40.395251 3411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:40.395572 3411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:40.395741 3411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:40.403136 3411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:40.403286 3411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:40.403460 3411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:57:40.413314 3411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35457
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:40.414649 3411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:40.416036 3411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:40.429379 3416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:40.429189 3419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:40.429191 3417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:40.430197 3411 server_base.cc:1034] running on GCE node
I20260621 06:57:40.430994 3411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:40.432021 3411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:40.433243 3411 hybrid_clock.cc:648] HybridClock initialized: now 1782025060433183 us; error 50 us; skew 500 ppm
I20260621 06:57:40.433555 3411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:40.436831 3411 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:57:40.438213 3411 webserver.cc:460] Webserver started at http://127.2.245.195:35255/ using document root <none> and password file <none>
I20260621 06:57:40.438874 3411 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:40.439042 3411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:40.439373 3411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:40.442175 3411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "9d5001af1614451a9bc44622ec485e63"
format_stamp: "Formatted at 2026-06-21 06:57:40 on dist-test-slave-9zdv"
server_key: "43e75be22bddab04150779be40a11b2d"
server_key_iv: "c30019749b48575ae1ea5b48a198fc1c"
server_key_version: "encryptionkey@0"
I20260621 06:57:40.443050 3411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "9d5001af1614451a9bc44622ec485e63"
format_stamp: "Formatted at 2026-06-21 06:57:40 on dist-test-slave-9zdv"
server_key: "43e75be22bddab04150779be40a11b2d"
server_key_iv: "c30019749b48575ae1ea5b48a198fc1c"
server_key_version: "encryptionkey@0"
I20260621 06:57:40.448087 3411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:57:40.451617 3426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:40.453044 3411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:57:40.453176 3411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "9d5001af1614451a9bc44622ec485e63"
format_stamp: "Formatted at 2026-06-21 06:57:40 on dist-test-slave-9zdv"
server_key: "43e75be22bddab04150779be40a11b2d"
server_key_iv: "c30019749b48575ae1ea5b48a198fc1c"
server_key_version: "encryptionkey@0"
I20260621 06:57:40.453353 3411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:40.466601 3411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:40.470721 3411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:40.471074 3411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:40.472010 3411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:40.473318 3411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:40.473392 3411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:40.473562 3411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:40.473618 3411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:40.507561 3411 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:35685
I20260621 06:57:40.507609 3538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:35685 every 8 connection(s)
I20260621 06:57:40.508800 3411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:57:40.512013 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3411
I20260621 06:57:40.512195 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:57:40.512635 3031 external_mini_cluster.cc:1428] Setting key 69cd71c801f7812e3f2d53946a8b3107
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:40.529351 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.511439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:55393 (local address 127.2.245.254:35457)
0621 06:57:40.511612 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.511616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.512997 (+ 1381us) server_negotiation.cc:407] Connection header received
0621 06:57:40.515117 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.515124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.515205 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.515313 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.516784 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.517515 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.519036 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.519438 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.519535 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.522810 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.522815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.522818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.522849 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.525529 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.526394 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.526400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.526403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.526574 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.527100 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.527105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.527108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.527573 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.527722 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.528761 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.528985 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:57:40.530249 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.511948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35457 (local address 127.2.245.195:55393)
0621 06:57:40.512820 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:40.512867 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:40.514781 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:40.515516 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:40.515525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:40.516106 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:40.516643 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:40.516659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.517633 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.517636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:40.518854 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:40.518866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.519626 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.519630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:40.519816 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.520486 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:40.520514 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:40.522614 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:40.525670 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:40.525678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:40.525692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:40.526251 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:40.526692 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:40.526697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:40.526700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:40.526930 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:40.527791 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:40.527799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:40.528382 (+ 583us) client_negotiation.cc:769] Sending connection context
0621 06:57:40.528682 (+ 300us) client_negotiation.cc:240] Negotiation successful
0621 06:57:40.529213 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":359,"threads_started":1}
I20260621 06:57:40.531620 3539 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35457
I20260621 06:57:40.531926 3539 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:40.532603 3539 heartbeater.cc:511] Master 127.2.245.254:35457 requested a full tablet report, sending...
I20260621 06:57:40.533891 3083 ts_manager.cc:194] Registered new tserver with Master: 9d5001af1614451a9bc44622ec485e63 (127.2.245.195:35685)
I20260621 06:57:40.534741 3083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:55393
I20260621 06:57:40.540719 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:57:40.559216 3272 heartbeater.cc:503] Master 127.2.245.254:35457 was elected leader, sending a full tablet report...
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:40.585049 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.570083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.2.245.193:35099)
0621 06:57:40.570697 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.570704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.570731 (+ 27us) server_negotiation.cc:407] Connection header received
0621 06:57:40.570839 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.570845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.571124 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.571253 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.572639 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.573684 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.574905 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.575544 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.575641 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.579584 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.579595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.579610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.579647 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.582484 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.583124 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.583130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.583132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.583249 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.583592 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.583595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.583597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.583909 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.584163 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.584519 (+ 356us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.584728 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":296,"threads_started":1}
W20260621 06:57:40.586267 3187 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48194
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:40.629204 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.615256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.2.245.193:35099)
0621 06:57:40.615480 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.615488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.615643 (+ 155us) server_negotiation.cc:407] Connection header received
0621 06:57:40.615800 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.615804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.615899 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.616037 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.617308 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.618029 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.619409 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.619814 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.619918 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.624054 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.624066 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.624070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.624101 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.626555 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.627306 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.627318 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.627321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.627443 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.627864 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.627868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.627870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.628110 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.628351 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.628642 (+ 291us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.628856 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 06:57:40.630752 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3144
I20260621 06:57:40.643096 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.631747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.2.245.193:35099)
0621 06:57:40.632003 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.632009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.632230 (+ 221us) server_negotiation.cc:407] Connection header received
0621 06:57:40.632336 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.632340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.632391 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.632464 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.633559 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.634252 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.635627 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.636041 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.636143 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.638204 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.638209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.638212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.638240 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.640886 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.641436 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.641439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.641441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.641576 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.641861 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.641864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.641866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.642117 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.642281 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.642518 (+ 237us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.642670 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260621 06:57:40.643960 3267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:40.728753 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3144
I20260621 06:57:40.747668 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3278
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:57:40.764232 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.748535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36980 (local address 127.2.245.194:45859)
0621 06:57:40.749130 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.749137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.749162 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:57:40.749222 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.749229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.749423 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.749629 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.750810 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.751968 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.753284 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.753839 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.753983 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.758191 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.758210 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.758224 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.758263 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.761210 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.761881 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.761888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.761891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.762075 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.762515 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.762521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.762524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.762879 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.763260 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.763604 (+ 344us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.763880 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":327,"threads_started":1}
I20260621 06:57:40.765580 3401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:40.849238 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3278
I20260621 06:57:40.868366 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3411
Jun 21 06:57:40 dist-test-slave-9zdv krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:57:40.884794 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:40.869382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.2.245.195:35685)
0621 06:57:40.869984 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:40.869992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:40.870022 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:57:40.870089 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:40.870096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:40.870311 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:40.870493 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:40.871839 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.872872 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.874158 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:40.874705 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:40.874842 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:40.878903 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:40.878916 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:40.878931 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:40.879017 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:40.882121 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.882792 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.882799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.882801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.882919 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:40.883503 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:40.883507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:40.883509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:40.883837 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:40.884092 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:40.884306 (+ 214us) server_negotiation.cc:299] Negotiation successful
0621 06:57:40.884496 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":313,"threads_started":1}
I20260621 06:57:40.886338 3534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:40.974650 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3411
I20260621 06:57:40.990751 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3051
I20260621 06:57:40.992046 3113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:41.094734 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3051
2026-06-21T06:57:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3895 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:57:41 dist-test-slave-9zdv krb5kdc[3560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:57:41 dist-test-slave-9zdv krb5kdc[3560](info): set up 2 sockets
Jun 21 06:57:41 dist-test-slave-9zdv krb5kdc[3560](info): commencing operation
krb5kdc: starting...
W20260621 06:57:42.163116 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:57:42 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:57:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:57:42.331411 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38101
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:32777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:42.641232 3573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:42.641563 3573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:42.641734 3573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:42.648689 3573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:57:42.648833 3573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:42.648929 3573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:42.649029 3573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:57:42.649113 3573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:57:42.660075 3573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:32777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:42.661368 3573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:42.662621 3573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:42.674204 3578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:42.674240 3579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:42.674283 3581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:42.675202 3573 server_base.cc:1034] running on GCE node
I20260621 06:57:42.675923 3573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:42.677227 3573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:42.678445 3573 hybrid_clock.cc:648] HybridClock initialized: now 1782025062678415 us; error 50 us; skew 500 ppm
I20260621 06:57:42.678791 3573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:42 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025062, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:42.685086 3573 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:57:42.686561 3573 webserver.cc:460] Webserver started at http://127.2.245.254:37845/ using document root <none> and password file <none>
I20260621 06:57:42.687343 3573 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:42.687438 3573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:42.687681 3573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:42.690493 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "e8d534bec8fe4def94b9b12d184415d5"
format_stamp: "Formatted at 2026-06-21 06:57:42 on dist-test-slave-9zdv"
server_key: "f357844ec8a8959cd5db64a831eebe1d"
server_key_iv: "d92afee366394e4aa4dc520a9befe5de"
server_key_version: "encryptionkey@0"
I20260621 06:57:42.691254 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "e8d534bec8fe4def94b9b12d184415d5"
format_stamp: "Formatted at 2026-06-21 06:57:42 on dist-test-slave-9zdv"
server_key: "f357844ec8a8959cd5db64a831eebe1d"
server_key_iv: "d92afee366394e4aa4dc520a9befe5de"
server_key_version: "encryptionkey@0"
I20260621 06:57:42.696182 3573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 06:57:42.699337 3588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:42.700582 3573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:57:42.700762 3573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "e8d534bec8fe4def94b9b12d184415d5"
format_stamp: "Formatted at 2026-06-21 06:57:42 on dist-test-slave-9zdv"
server_key: "f357844ec8a8959cd5db64a831eebe1d"
server_key_iv: "d92afee366394e4aa4dc520a9befe5de"
server_key_version: "encryptionkey@0"
I20260621 06:57:42.700927 3573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:42.712958 3573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:42.716794 3573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:42.717092 3573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:42.735172 3573 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:38101
I20260621 06:57:42.735190 3639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:38101 every 8 connection(s)
I20260621 06:57:42.736507 3573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:57:42.737154 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3573
I20260621 06:57:42.737394 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:57:42.738153 3031 external_mini_cluster.cc:1428] Setting key d97dae64e282bfb6fff14e821bc49437
I20260621 06:57:42.740806 3640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 06:57:42 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:42.753474 3640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5: Bootstrap starting.
I20260621 06:57:42.756493 3640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:42.757730 3640 log.cc:824] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:42.760581 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:42.740069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.2.245.254:38101)
0621 06:57:42.740847 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:42.740858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:42.740898 (+ 40us) server_negotiation.cc:407] Connection header received
0621 06:57:42.742502 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:42.742523 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:42.742884 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:42.743307 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:42.745028 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:42.746386 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:42.748106 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:42.748720 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:42.748857 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:42.753493 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:42.753501 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:42.753515 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:42.753549 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:42.756554 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:42.757378 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:42.757385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:42.757392 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:42.757547 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:42.757958 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:42.757961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:42.757963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:42.758610 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:42.758918 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:42.759285 (+ 367us) server_negotiation.cc:299] Negotiation successful
0621 06:57:42.759642 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":357,"threads_started":1}
I20260621 06:57:42.760666 3640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5: No bootstrap required, opened a new log
I20260621 06:57:42.768465 3640 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } }
I20260621 06:57:42.768849 3640 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:42.768955 3640 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d534bec8fe4def94b9b12d184415d5, State: Initialized, Role: FOLLOWER
I20260621 06:57:42.769548 3640 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } }
I20260621 06:57:42.769742 3640 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:57:42.769889 3640 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:57:42.770052 3640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:42.772148 3640 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } }
I20260621 06:57:42.772593 3640 leader_election.cc:302] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8d534bec8fe4def94b9b12d184415d5; no voters:
I20260621 06:57:42.773206 3640 leader_election.cc:288] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:57:42.773352 3645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:42.774021 3645 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [term 1 LEADER]: Becoming Leader. State: Replica: e8d534bec8fe4def94b9b12d184415d5, State: Running, Role: LEADER
I20260621 06:57:42.774610 3645 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } }
I20260621 06:57:42.775332 3640 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:57:42.777392 3646 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8d534bec8fe4def94b9b12d184415d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } } }
I20260621 06:57:42.777438 3647 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8d534bec8fe4def94b9b12d184415d5. Latest consensus state: current_term: 1 leader_uuid: "e8d534bec8fe4def94b9b12d184415d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d534bec8fe4def94b9b12d184415d5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38101 } } }
I20260621 06:57:42.777814 3647 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:42.777814 3646 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:42.780236 3652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:57:42.784544 3652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:57:42.792774 3652 catalog_manager.cc:1347] Generated new cluster ID: cdd495ad7b264dfeb1c814cdb4c59e29
I20260621 06:57:42.792896 3652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:57:42.801950 3652 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:57:42.803723 3652 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:57:42.817226 3652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8d534bec8fe4def94b9b12d184415d5: Generated new TSK 0
I20260621 06:57:42.818179 3652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:57:42.866432 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:32777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:43.181216 3666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:43.181475 3666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:43.181685 3666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:43.188846 3666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:43.188997 3666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:43.189196 3666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:57:43.198660 3666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:32777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:43.199946 3666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:43.201236 3666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:43.214684 3671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:43.214633 3674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:43.214658 3672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:43.215585 3666 server_base.cc:1034] running on GCE node
I20260621 06:57:43.216255 3666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:43.217168 3666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:43.218407 3666 hybrid_clock.cc:648] HybridClock initialized: now 1782025063218359 us; error 41 us; skew 500 ppm
I20260621 06:57:43.218730 3666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:43 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:43.221817 3666 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:57:43.223359 3666 webserver.cc:460] Webserver started at http://127.2.245.193:46135/ using document root <none> and password file <none>
I20260621 06:57:43.224042 3666 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:43.224185 3666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:43.224490 3666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:43.227449 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "8d21882c040e4c748c834a63fc5ac341"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "6cc212cc2bbd95cac24e02b414d26782"
server_key_iv: "1eb8d043e74246d8770227bd8e1af91c"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.228258 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "8d21882c040e4c748c834a63fc5ac341"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "6cc212cc2bbd95cac24e02b414d26782"
server_key_iv: "1eb8d043e74246d8770227bd8e1af91c"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.233402 3666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260621 06:57:43.236766 3681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.238015 3666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260621 06:57:43.238194 3666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "8d21882c040e4c748c834a63fc5ac341"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "6cc212cc2bbd95cac24e02b414d26782"
server_key_iv: "1eb8d043e74246d8770227bd8e1af91c"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.238384 3666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:43.274116 3666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:43.278409 3666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:43.278743 3666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:43.279850 3666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:43.281201 3666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:43.281306 3666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.281447 3666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:43.281517 3666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.316843 3666 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:40585
I20260621 06:57:43.316861 3793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:40585 every 8 connection(s)
I20260621 06:57:43.318310 3666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:57:43.323436 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3666
I20260621 06:57:43.323654 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:57:43.324409 3031 external_mini_cluster.cc:1428] Setting key 46e838e60197bfe0e864289e3ef84da8
Jun 21 06:57:43 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:43.338585 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:43.320883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48317 (local address 127.2.245.254:38101)
0621 06:57:43.321315 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:43.321320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:43.322121 (+ 801us) server_negotiation.cc:407] Connection header received
0621 06:57:43.324562 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:43.324567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:43.324634 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:43.324741 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:43.326266 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.327037 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.328560 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.328933 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.329071 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:43.332544 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:43.332550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:43.332555 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:43.332587 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:43.335224 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:43.336031 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:43.336038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:43.336045 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:43.336206 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:43.336659 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:43.336666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:43.336671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:43.337084 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:43.337263 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:43.337914 (+ 651us) server_negotiation.cc:299] Negotiation successful
0621 06:57:43.338174 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":236,"threads_started":1}
I20260621 06:57:43.339305 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:43.321323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38101 (local address 127.2.245.193:48317)
0621 06:57:43.321958 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:43.322000 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:43.324218 (+ 2218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:43.325010 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:43.325023 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:43.325546 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:43.326095 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:43.326111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.327174 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.327186 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:43.328394 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:43.328405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.329165 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.329173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:43.329392 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:43.330117 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:43.330137 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:43.332324 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:43.335365 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:43.335374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:43.335389 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:43.335884 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:43.336329 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:43.336332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:43.336335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:43.336519 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:43.337208 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:43.337213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:43.337523 (+ 310us) client_negotiation.cc:769] Sending connection context
0621 06:57:43.337768 (+ 245us) client_negotiation.cc:240] Negotiation successful
0621 06:57:43.338135 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":253,"threads_started":1}
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:57:43.340871 3794 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38101
I20260621 06:57:43.341248 3794 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:43.341997 3794 heartbeater.cc:511] Master 127.2.245.254:38101 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:57:43.344563 3605 ts_manager.cc:194] Registered new tserver with Master: 8d21882c040e4c748c834a63fc5ac341 (127.2.245.193:40585)
I20260621 06:57:43.346730 3605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:48317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:57:43.363859 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:32777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:43.674759 3800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:43.675079 3800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:43.675254 3800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:43.682541 3800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:43.682646 3800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:43.682763 3800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:57:43.692858 3800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:32777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:43.694646 3800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:43.696410 3800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:43.709664 3806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:43.709666 3805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:43.709600 3808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:43.710867 3800 server_base.cc:1034] running on GCE node
I20260621 06:57:43.711606 3800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:43.712527 3800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:43.713754 3800 hybrid_clock.cc:648] HybridClock initialized: now 1782025063713721 us; error 52 us; skew 500 ppm
I20260621 06:57:43.714072 3800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:43 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:43.717288 3800 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:57:43.718717 3800 webserver.cc:460] Webserver started at http://127.2.245.194:35473/ using document root <none> and password file <none>
I20260621 06:57:43.719460 3800 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:43.719602 3800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:43.719926 3800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:43.722762 3800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "04ddfc1d85404b5088f10c774a351c1a"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "dcdc487b4aadab98de75b3d14b8ba376"
server_key_iv: "92b5fdef6722ffe9c2112c4f0e82f425"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.723728 3800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "04ddfc1d85404b5088f10c774a351c1a"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "dcdc487b4aadab98de75b3d14b8ba376"
server_key_iv: "92b5fdef6722ffe9c2112c4f0e82f425"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.728672 3800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:57:43.731988 3815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.733242 3800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:57:43.733359 3800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "04ddfc1d85404b5088f10c774a351c1a"
format_stamp: "Formatted at 2026-06-21 06:57:43 on dist-test-slave-9zdv"
server_key: "dcdc487b4aadab98de75b3d14b8ba376"
server_key_iv: "92b5fdef6722ffe9c2112c4f0e82f425"
server_key_version: "encryptionkey@0"
I20260621 06:57:43.733522 3800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:43.745049 3800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:43.749157 3800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:43.749459 3800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:43.750385 3800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:43.751750 3800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:43.751821 3800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.751968 3800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:43.752044 3800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:43.784277 3800 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:37099
I20260621 06:57:43.784322 3927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:37099 every 8 connection(s)
I20260621 06:57:43.785523 3800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:57:43.788918 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3800
I20260621 06:57:43.789110 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:57:43.789582 3031 external_mini_cluster.cc:1428] Setting key f6f66251608781b2f45f99fb61a1895c
Jun 21 06:57:43 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:57:43.806994 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:43.788138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:34393 (local address 127.2.245.254:38101)
0621 06:57:43.788302 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:43.788306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:43.789546 (+ 1240us) server_negotiation.cc:407] Connection header received
0621 06:57:43.791713 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:43.791719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:43.791806 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:43.791911 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:43.793313 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.794005 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.795753 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.796181 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.796294 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:43.800303 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:43.800308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:43.800311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:43.800342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:43.803739 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:43.804502 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:43.804508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:43.804512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:43.804714 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:43.805120 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:43.805126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:43.805129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:43.805529 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:43.805735 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:43.806395 (+ 660us) server_negotiation.cc:299] Negotiation successful
0621 06:57:43.806626 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 06:57:43.807935 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:43.788602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38101 (local address 127.2.245.194:34393)
0621 06:57:43.789374 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:43.789426 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:43.791455 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:43.792131 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:43.792138 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:43.792625 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:43.793144 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:43.793170 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.794156 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.794180 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:43.795577 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:43.795600 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:43.796414 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:43.796422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:43.796742 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:43.797426 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:43.797457 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:43.800056 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:43.803891 (+ 3835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:43.803897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:43.803909 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:43.804372 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:43.804824 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:43.804828 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:43.804830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:43.804996 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:43.805723 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:43.805734 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:43.806032 (+ 298us) client_negotiation.cc:769] Sending connection context
0621 06:57:43.806289 (+ 257us) client_negotiation.cc:240] Negotiation successful
0621 06:57:43.806813 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":319,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
I20260621 06:57:43.809357 3928 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38101
I20260621 06:57:43.809742 3928 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:43.810583 3928 heartbeater.cc:511] Master 127.2.245.254:38101 requested a full tablet report, sending...
I20260621 06:57:43.812270 3605 ts_manager.cc:194] Registered new tserver with Master: 04ddfc1d85404b5088f10c774a351c1a (127.2.245.194:37099)
I20260621 06:57:43.813341 3605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:34393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:57:43.829712 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:32777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:44.137607 3933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:44.137902 3933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:44.138089 3933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:44.145447 3933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:44.145597 3933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:44.145790 3933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:57:44.155746 3933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:32777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38101
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:44.157032 3933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:44.158347 3933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:44.171944 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:44.171981 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:44.172030 3941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:44.172842 3933 server_base.cc:1034] running on GCE node
I20260621 06:57:44.173658 3933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:44.174678 3933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:44.175904 3933 hybrid_clock.cc:648] HybridClock initialized: now 1782025064175855 us; error 44 us; skew 500 ppm
I20260621 06:57:44.176229 3933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:44.179551 3933 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:57:44.181165 3933 webserver.cc:460] Webserver started at http://127.2.245.195:40925/ using document root <none> and password file <none>
I20260621 06:57:44.181843 3933 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:44.181972 3933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:44.182296 3933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:44.185196 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "333af74bdbdd4da6b834180d8669598c"
format_stamp: "Formatted at 2026-06-21 06:57:44 on dist-test-slave-9zdv"
server_key: "086a8a7f6fd770b35b229dc8f65e40a8"
server_key_iv: "eb2e3a38366ca8daf5b09fff82a62d5f"
server_key_version: "encryptionkey@0"
I20260621 06:57:44.186141 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "333af74bdbdd4da6b834180d8669598c"
format_stamp: "Formatted at 2026-06-21 06:57:44 on dist-test-slave-9zdv"
server_key: "086a8a7f6fd770b35b229dc8f65e40a8"
server_key_iv: "eb2e3a38366ca8daf5b09fff82a62d5f"
server_key_version: "encryptionkey@0"
I20260621 06:57:44.191414 3933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:57:44.194667 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:44.196026 3933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:57:44.196218 3933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "333af74bdbdd4da6b834180d8669598c"
format_stamp: "Formatted at 2026-06-21 06:57:44 on dist-test-slave-9zdv"
server_key: "086a8a7f6fd770b35b229dc8f65e40a8"
server_key_iv: "eb2e3a38366ca8daf5b09fff82a62d5f"
server_key_version: "encryptionkey@0"
I20260621 06:57:44.196411 3933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:44.218307 3933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:44.222424 3933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:44.222774 3933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:44.223954 3933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:57:44.225334 3933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:44.225466 3933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:44.225601 3933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:44.225728 3933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:44.258528 3933 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:45351
I20260621 06:57:44.258582 4060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:45351 every 8 connection(s)
I20260621 06:57:44.259858 3933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:57:44.264882 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 3933
I20260621 06:57:44.265062 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:57:44.265605 3031 external_mini_cluster.cc:1428] Setting key 2240a05545fd5a997108b7e2dc746a82
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:44.280586 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.262418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:41241 (local address 127.2.245.254:38101)
0621 06:57:44.262592 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.262596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.263739 (+ 1143us) server_negotiation.cc:407] Connection header received
0621 06:57:44.266172 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.266178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.266268 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.266392 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.268357 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.269230 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.270710 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.271170 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.271265 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.274925 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.274929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.274932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.275008 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.277604 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.278214 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.278217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.278219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.278345 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.278738 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.278742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.278744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.279084 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.279241 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.279950 (+ 709us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.280224 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 06:57:44.281510 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.262849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38101 (local address 127.2.245.195:41241)
0621 06:57:44.263587 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.263623 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.265806 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.266673 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.266684 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.267513 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:44.268184 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.268201 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.269358 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.269362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.270573 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.270584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.271375 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.271381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.271777 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.272466 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:44.272487 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:44.274734 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:44.277728 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:44.277736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:44.277750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:44.278120 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:44.278430 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:44.278433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:44.278435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:44.278599 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:44.279219 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:44.279227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:44.279568 (+ 341us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.279834 (+ 266us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.280428 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":271,"threads_started":1}
I20260621 06:57:44.282768 4061 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38101
I20260621 06:57:44.283095 4061 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:44.283772 4061 heartbeater.cc:511] Master 127.2.245.254:38101 requested a full tablet report, sending...
I20260621 06:57:44.285132 3605 ts_manager.cc:194] Registered new tserver with Master: 333af74bdbdd4da6b834180d8669598c (127.2.245.195:45351)
I20260621 06:57:44.285912 3605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:41241
I20260621 06:57:44.292992 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:57:44.311908 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.299010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.2.245.254:38101)
0621 06:57:44.299163 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.299168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.299728 (+ 560us) server_negotiation.cc:407] Connection header received
0621 06:57:44.299910 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.299916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.299993 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.300092 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.301576 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.302297 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.303633 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.304089 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.304209 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.306506 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.306512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.306516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.306551 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.309305 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.309964 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.309970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.309974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.310112 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.310480 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.310488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.310493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.310796 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.311066 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.311397 (+ 331us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.311612 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 06:57:44.318264 3605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:44.325279 3605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:44.349730 3794 heartbeater.cc:503] Master 127.2.245.254:38101 was elected leader, sending a full tablet report...
I20260621 06:57:44.360844 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.337734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.2.245.194:37099)
0621 06:57:44.349401 (+ 11667us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.349412 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.350346 (+ 934us) server_negotiation.cc:407] Connection header received
0621 06:57:44.350473 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.350484 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.350806 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.351930 (+ 1124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:44.352205 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.354226 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.357403 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.359149 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.359400 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.359603 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.360320 (+ 717us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.360491 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11452,"thread_start_us":11306,"threads_started":1}
I20260621 06:57:44.361038 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.338116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.2.245.193:40585)
0621 06:57:44.349930 (+ 11814us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.349937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.350722 (+ 785us) server_negotiation.cc:407] Connection header received
0621 06:57:44.350815 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.350822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.351082 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.351329 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:44.352503 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.354567 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.357813 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.359519 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.359652 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.359902 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.360540 (+ 638us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.360697 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11645,"thread_start_us":11811,"threads_started":1}
I20260621 06:57:44.360942 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.337721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:40585 (local address 127.0.0.1:34840)
0621 06:57:44.350459 (+ 12738us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.350477 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.350588 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.351631 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.351639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.351667 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:44.352262 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.352276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.354786 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.354795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.357635 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.357649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.360037 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.360042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.360237 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.360267 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.360357 (+ 90us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.360492 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12619,"mutex_wait_us":92,"thread_start_us":12572,"threads_started":1}
I20260621 06:57:44.361586 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.337411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:37099 (local address 127.0.0.1:46138)
0621 06:57:44.349858 (+ 12447us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.349906 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.350075 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.351112 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.351123 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.351169 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:44.351649 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.351664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.354415 (+ 2751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.354423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.357276 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.357287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.359763 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.359770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.360026 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.360098 (+ 72us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.360251 (+ 153us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.361217 (+ 966us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12179,"mutex_wait_us":787,"thread_start_us":12015,"threads_started":1}
I20260621 06:57:44.363566 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.340251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:45351 (local address 127.0.0.1:38354)
0621 06:57:44.355725 (+ 15474us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.355743 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.355873 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.356755 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.356760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.356783 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:44.357180 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.357191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.358663 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.358669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.360645 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.360660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.362699 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.362708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.362898 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.362963 (+ 65us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.363070 (+ 107us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.363189 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15053,"mutex_wait_us":61,"thread_start_us":14866,"threads_started":1}
I20260621 06:57:44.364954 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd210540d174630891807789f732319 (DEFAULT_TABLE table=test-table [id=f734549650544ddd9503910de9318553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:44.364950 3729 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd210540d174630891807789f732319 (DEFAULT_TABLE table=test-table [id=f734549650544ddd9503910de9318553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:44.366081 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd210540d174630891807789f732319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:44.366212 3729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd210540d174630891807789f732319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:44.369266 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.340926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.2.245.195:45351)
0621 06:57:44.355744 (+ 14818us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.355752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.355864 (+ 112us) server_negotiation.cc:407] Connection header received
0621 06:57:44.356077 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.356085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.356304 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.356469 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:44.357380 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.358521 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.360827 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.362491 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.368470 (+ 5979us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.368699 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.368766 (+ 67us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.368944 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14643,"thread_start_us":14493,"threads_started":1}
I20260621 06:57:44.372792 3996 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd210540d174630891807789f732319 (DEFAULT_TABLE table=test-table [id=f734549650544ddd9503910de9318553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:44.374250 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd210540d174630891807789f732319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:44.377643 4079 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Bootstrap starting.
I20260621 06:57:44.379576 4080 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Bootstrap starting.
I20260621 06:57:44.381493 4079 tablet_bootstrap.cc:654] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:44.382859 4079 log.cc:824] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:44.383049 4080 tablet_bootstrap.cc:654] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:44.384112 4080 log.cc:824] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:44.385905 4079 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: No bootstrap required, opened a new log
I20260621 06:57:44.386188 4079 ts_tablet_manager.cc:1397] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260621 06:57:44.387017 4080 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: No bootstrap required, opened a new log
I20260621 06:57:44.387270 4080 ts_tablet_manager.cc:1397] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260621 06:57:44.388156 4083 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Bootstrap starting.
I20260621 06:57:44.391450 4083 tablet_bootstrap.cc:654] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:44.392412 4083 log.cc:824] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:44.395006 4083 tablet_bootstrap.cc:492] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: No bootstrap required, opened a new log
I20260621 06:57:44.395270 4083 ts_tablet_manager.cc:1397] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260621 06:57:44.397467 4079 raft_consensus.cc:348] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.397701 4080 raft_consensus.cc:348] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.397961 4079 raft_consensus.cc:374] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:44.398061 4079 raft_consensus.cc:729] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ddfc1d85404b5088f10c774a351c1a, State: Initialized, Role: FOLLOWER
I20260621 06:57:44.398097 4080 raft_consensus.cc:374] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:44.398190 4080 raft_consensus.cc:729] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d21882c040e4c748c834a63fc5ac341, State: Initialized, Role: FOLLOWER
I20260621 06:57:44.398707 4079 consensus_queue.cc:260] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.398774 4080 consensus_queue.cc:260] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.400719 4079 ts_tablet_manager.cc:1428] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260621 06:57:44.400919 4080 ts_tablet_manager.cc:1428] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260621 06:57:44.401257 3928 heartbeater.cc:503] Master 127.2.245.254:38101 was elected leader, sending a full tablet report...
I20260621 06:57:44.403964 4086 raft_consensus.cc:482] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:44.404187 4086 raft_consensus.cc:504] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.404263 4083 raft_consensus.cc:348] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.404696 4083 raft_consensus.cc:374] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:44.404785 4083 raft_consensus.cc:729] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333af74bdbdd4da6b834180d8669598c, State: Initialized, Role: FOLLOWER
I20260621 06:57:44.405489 4083 consensus_queue.cc:260] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.411149 4086 leader_election.cc:288] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d21882c040e4c748c834a63fc5ac341 (127.2.245.193:40585), 333af74bdbdd4da6b834180d8669598c (127.2.245.195:45351)
I20260621 06:57:44.411705 4061 heartbeater.cc:503] Master 127.2.245.254:38101 was elected leader, sending a full tablet report...
I20260621 06:57:44.413142 4083 ts_tablet_manager.cc:1428] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260621 06:57:44.417688 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.406741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:53327 (local address 127.2.245.193:40585)
0621 06:57:44.406919 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.406925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.410045 (+ 3120us) server_negotiation.cc:407] Connection header received
0621 06:57:44.411094 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.411100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.411179 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.411273 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:44.412028 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.413061 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.415681 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.416847 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.416971 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.417021 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.417430 (+ 409us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.417519 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 06:57:44.417891 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.406158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:40585 (local address 127.2.245.194:53327)
0621 06:57:44.409925 (+ 3767us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.409942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.410113 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.411444 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.411449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.411476 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:44.411867 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.411884 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.413235 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.413241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.415517 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.415532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.417064 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.417070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.417261 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.417297 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.417402 (+ 105us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.417532 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3649,"thread_start_us":3435,"threads_started":1}
I20260621 06:57:44.418851 3749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd210540d174630891807789f732319" candidate_uuid: "04ddfc1d85404b5088f10c774a351c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d21882c040e4c748c834a63fc5ac341" is_pre_election: true
I20260621 06:57:44.419495 3749 raft_consensus.cc:2454] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ddfc1d85404b5088f10c774a351c1a in term 0.
I20260621 06:57:44.420135 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.406353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:57841 (local address 127.2.245.195:45351)
0621 06:57:44.406601 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.406610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.413634 (+ 7024us) server_negotiation.cc:407] Connection header received
0621 06:57:44.413950 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.413955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.414021 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.414080 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:44.414866 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.415845 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.418024 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.419174 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.419368 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.419431 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.419741 (+ 310us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.419873 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 06:57:44.420280 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.406143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:45351 (local address 127.2.245.194:57841)
0621 06:57:44.413533 (+ 7390us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:44.413553 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:44.413738 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:44.414314 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:44.414318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:44.414342 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:44.414709 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.414717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.416015 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.416020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.417803 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:44.417817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.419432 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.419437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:44.419561 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.419583 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 06:57:44.419704 (+ 121us) client_negotiation.cc:240] Negotiation successful
0621 06:57:44.419843 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7231,"mutex_wait_us":471,"thread_start_us":4161,"threads_started":1}
I20260621 06:57:44.420223 3818 leader_election.cc:302] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ddfc1d85404b5088f10c774a351c1a, 8d21882c040e4c748c834a63fc5ac341; no voters:
I20260621 06:57:44.420647 4086 raft_consensus.cc:2781] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:44.420799 4086 raft_consensus.cc:482] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:44.420899 4086 raft_consensus.cc:3037] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:44.421238 4016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd210540d174630891807789f732319" candidate_uuid: "04ddfc1d85404b5088f10c774a351c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333af74bdbdd4da6b834180d8669598c" is_pre_election: true
I20260621 06:57:44.421610 4016 raft_consensus.cc:2454] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ddfc1d85404b5088f10c774a351c1a in term 0.
I20260621 06:57:44.423213 4086 raft_consensus.cc:504] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.423839 4086 leader_election.cc:288] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [CANDIDATE]: Term 1 election: Requested vote from peers 8d21882c040e4c748c834a63fc5ac341 (127.2.245.193:40585), 333af74bdbdd4da6b834180d8669598c (127.2.245.195:45351)
I20260621 06:57:44.424233 3749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd210540d174630891807789f732319" candidate_uuid: "04ddfc1d85404b5088f10c774a351c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d21882c040e4c748c834a63fc5ac341"
I20260621 06:57:44.424443 3749 raft_consensus.cc:3037] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:44.424437 4016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd210540d174630891807789f732319" candidate_uuid: "04ddfc1d85404b5088f10c774a351c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333af74bdbdd4da6b834180d8669598c"
I20260621 06:57:44.424630 4016 raft_consensus.cc:3037] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:44.426641 3749 raft_consensus.cc:2454] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ddfc1d85404b5088f10c774a351c1a in term 1.
I20260621 06:57:44.426854 4016 raft_consensus.cc:2454] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ddfc1d85404b5088f10c774a351c1a in term 1.
I20260621 06:57:44.427073 3818 leader_election.cc:302] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ddfc1d85404b5088f10c774a351c1a, 8d21882c040e4c748c834a63fc5ac341; no voters:
I20260621 06:57:44.427346 4086 raft_consensus.cc:2781] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:44.427719 4086 raft_consensus.cc:686] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [term 1 LEADER]: Becoming Leader. State: Replica: 04ddfc1d85404b5088f10c774a351c1a, State: Running, Role: LEADER
I20260621 06:57:44.428169 4086 consensus_queue.cc:237] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } }
I20260621 06:57:44.432682 3605 catalog_manager.cc:5495] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 04ddfc1d85404b5088f10c774a351c1a (127.2.245.194). New cstate: current_term: 1 leader_uuid: "04ddfc1d85404b5088f10c774a351c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ddfc1d85404b5088f10c774a351c1a" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 37099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d21882c040e4c748c834a63fc5ac341" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "333af74bdbdd4da6b834180d8669598c" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45351 } health_report { overall_health: UNKNOWN } } }
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:44.466703 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.451946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.2.245.193:40585)
0621 06:57:44.452129 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.452135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.452640 (+ 505us) server_negotiation.cc:407] Connection header received
0621 06:57:44.452909 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.452914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.452988 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.453099 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.454574 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.455429 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.456673 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.457095 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.457254 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.461355 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.461368 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.461381 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.461413 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.464273 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.464894 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.464900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.464903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.465016 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.465346 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.465349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.465351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.465692 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.465932 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.466159 (+ 227us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.466308 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
W20260621 06:57:44.511482 4062 tablet.cc:2369] T 4fd210540d174630891807789f732319 P 333af74bdbdd4da6b834180d8669598c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:57:44.513464 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.499520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.2.245.193:40585)
0621 06:57:44.499751 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.499789 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.500056 (+ 267us) server_negotiation.cc:407] Connection header received
0621 06:57:44.500265 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.500271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.500340 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.500450 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.501839 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.502586 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.503918 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.504382 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.504513 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.508445 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.508449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.508453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.508483 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.511079 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.511691 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.511696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.511699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.511864 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.512251 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.512256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.512259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.512542 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.512747 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.512951 (+ 204us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.513113 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260621 06:57:44.537510 3929 tablet.cc:2369] T 4fd210540d174630891807789f732319 P 04ddfc1d85404b5088f10c774a351c1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:44.557425 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.543531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.2.245.193:40585)
0621 06:57:44.543892 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.543901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.544300 (+ 399us) server_negotiation.cc:407] Connection header received
0621 06:57:44.544521 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.544525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.544586 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.544687 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.546115 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.546771 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.547989 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.548413 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.548554 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.552314 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.552319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.552322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.552362 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.554809 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.555480 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.555486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.555490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.555619 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.556004 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.556009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.556012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.556293 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.556550 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.556856 (+ 306us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.557083 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260621 06:57:44.559204 3709 tablet_service.cc:2908] Rejecting scan request for tablet 4fd210540d174630891807789f732319: Uninitialized: clean time has not yet been initialized
I20260621 06:57:44.562618 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3666
W20260621 06:57:44.570715 3795 tablet.cc:2369] T 4fd210540d174630891807789f732319 P 8d21882c040e4c748c834a63fc5ac341: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:57:44.575316 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.563492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.2.245.193:40585)
0621 06:57:44.563669 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.563673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.564033 (+ 360us) server_negotiation.cc:407] Connection header received
0621 06:57:44.564165 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.564169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.564234 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.564320 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.565373 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.566002 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.567191 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.567597 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.567699 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.569679 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.569683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.569686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.569714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.572451 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.573161 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.573168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.573173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.573332 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.573785 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.573828 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.573834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.574176 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.574475 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.574754 (+ 279us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.574983 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 06:57:44.575922 3789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:44.663513 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3666
I20260621 06:57:44.680603 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3800
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:57:44.695088 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.681522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33122 (local address 127.2.245.194:37099)
0621 06:57:44.681717 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.681721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.681739 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:57:44.681795 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.681799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.681864 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.681945 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.683328 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.684087 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.685328 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.685713 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.685849 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.689804 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.689818 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.689853 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.689894 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.692642 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.693280 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.693287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.693289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.693423 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.693731 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.693734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.693736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.694071 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.694282 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.694534 (+ 252us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.694702 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 06:57:44.695809 3923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:44.788275 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3800
I20260621 06:57:44.804869 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3933
Jun 21 06:57:44 dist-test-slave-9zdv krb5kdc[3560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:57:44.819442 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:44.805621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.2.245.195:45351)
0621 06:57:44.805789 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:44.805793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:44.805811 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:57:44.806005 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:44.806009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:44.806077 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:44.806161 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:44.807404 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.808109 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.809267 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:44.809650 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:44.809737 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:44.813926 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:44.813939 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:44.813955 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:44.813988 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:44.816553 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.817223 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.817230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.817232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.817393 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:44.817791 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:44.817798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:44.817802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:44.818164 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:44.818484 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:44.818760 (+ 276us) server_negotiation.cc:299] Negotiation successful
0621 06:57:44.819111 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 06:57:44.820073 4056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:44.906618 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3933
I20260621 06:57:44.922710 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 3573
I20260621 06:57:44.923979 3635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:45.031711 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 3573
2026-06-21T06:57:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3936 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:57:45 dist-test-slave-9zdv krb5kdc[4109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:57:45 dist-test-slave-9zdv krb5kdc[4109](info): set up 2 sockets
Jun 21 06:57:45 dist-test-slave-9zdv krb5kdc[4109](info): commencing operation
krb5kdc: starting...
W20260621 06:57:46.123306 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:57:46 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:57:46.305740 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34549
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:34549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:46.619998 4122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:46.620311 4122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:46.620398 4122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:46.627336 4122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:57:46.627439 4122 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 06:57:46.627477 4122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:46.627532 4122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:46.627579 4122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:57:46.627632 4122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:57:46.637108 4122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:34549
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:46.638286 4122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:46.639526 4122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:46.651175 4130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:46.651264 4128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:46.651197 4127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:46.652400 4122 server_base.cc:1034] running on GCE node
I20260621 06:57:46.653146 4122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:46.654515 4122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:46.655753 4122 hybrid_clock.cc:648] HybridClock initialized: now 1782025066655689 us; error 41 us; skew 500 ppm
I20260621 06:57:46.656085 4122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:46 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:46.659283 4122 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:57:46.660774 4122 webserver.cc:460] Webserver started at http://127.2.245.254:41553/ using document root <none> and password file <none>
I20260621 06:57:46.661482 4122 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:46.661613 4122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:46.661931 4122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:46.664886 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "f89cc0189b594571b42a01d8baeee6eb"
format_stamp: "Formatted at 2026-06-21 06:57:46 on dist-test-slave-9zdv"
server_key: "1a4d77f5abbb50409f097a9f7e30f638"
server_key_iv: "736bb4dc3cb43d91f191f2f5c3ab0f60"
server_key_version: "encryptionkey@0"
I20260621 06:57:46.665678 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "f89cc0189b594571b42a01d8baeee6eb"
format_stamp: "Formatted at 2026-06-21 06:57:46 on dist-test-slave-9zdv"
server_key: "1a4d77f5abbb50409f097a9f7e30f638"
server_key_iv: "736bb4dc3cb43d91f191f2f5c3ab0f60"
server_key_version: "encryptionkey@0"
I20260621 06:57:46.671902 4122 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260621 06:57:46.675321 4137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:46.676712 4122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 06:57:46.676924 4122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "f89cc0189b594571b42a01d8baeee6eb"
format_stamp: "Formatted at 2026-06-21 06:57:46 on dist-test-slave-9zdv"
server_key: "1a4d77f5abbb50409f097a9f7e30f638"
server_key_iv: "736bb4dc3cb43d91f191f2f5c3ab0f60"
server_key_version: "encryptionkey@0"
I20260621 06:57:46.677181 4122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:46.694119 4122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:46.698477 4122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:46.698760 4122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:46.721483 4122 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:34549
I20260621 06:57:46.721490 4198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:34549 every 8 connection(s)
I20260621 06:57:46.722890 4122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:57:46.726660 4199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:46.732535 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4122
I20260621 06:57:46.732755 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:57:46.733551 3031 external_mini_cluster.cc:1428] Setting key 30675ddf81917a6ab52350b5541adc12
I20260621 06:57:46.737600 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Bootstrap starting.
I20260621 06:57:46.741494 4199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:46.742709 4199 log.cc:824] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:46.745885 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: No bootstrap required, opened a new log
Jun 21 06:57:46 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:46.753855 4199 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } }
I20260621 06:57:46.754225 4199 raft_consensus.cc:374] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:46.754359 4199 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cc0189b594571b42a01d8baeee6eb, State: Initialized, Role: FOLLOWER
I20260621 06:57:46.755072 4199 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } }
I20260621 06:57:46.755376 4199 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:57:46.755517 4199 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:57:46.755728 4199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:46.756856 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:46.735638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.2.245.254:34549)
0621 06:57:46.736443 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:46.736454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:46.736493 (+ 39us) server_negotiation.cc:407] Connection header received
0621 06:57:46.738091 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:46.738112 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:46.738470 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:46.738773 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:46.740569 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:46.741974 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:46.743743 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:46.744552 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:46.744690 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:46.749355 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:46.749363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:46.749378 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:46.749418 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:46.752533 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:46.753243 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:46.753250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:46.753255 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:46.753406 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:46.753769 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:46.753779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:46.753781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:46.754410 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:46.754727 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:46.755206 (+ 479us) server_negotiation.cc:299] Negotiation successful
0621 06:57:46.755814 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":379,"threads_started":1}
I20260621 06:57:46.758493 4199 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } }
I20260621 06:57:46.759064 4199 leader_election.cc:302] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89cc0189b594571b42a01d8baeee6eb; no voters:
I20260621 06:57:46.759711 4199 leader_election.cc:288] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:57:46.759838 4204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:46.760560 4204 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [term 1 LEADER]: Becoming Leader. State: Replica: f89cc0189b594571b42a01d8baeee6eb, State: Running, Role: LEADER
I20260621 06:57:46.761169 4204 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } }
I20260621 06:57:46.762039 4199 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:57:46.765996 4206 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89cc0189b594571b42a01d8baeee6eb. Latest consensus state: current_term: 1 leader_uuid: "f89cc0189b594571b42a01d8baeee6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } } }
I20260621 06:57:46.766464 4206 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:46.766544 4205 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89cc0189b594571b42a01d8baeee6eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89cc0189b594571b42a01d8baeee6eb" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34549 } } }
I20260621 06:57:46.767031 4205 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:46.772116 4212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:57:46.776360 4212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:57:46.784749 4212 catalog_manager.cc:1347] Generated new cluster ID: 6ec764a7e6124d2fb07718cd2a138888
I20260621 06:57:46.784864 4212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:57:46.806596 4212 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:57:46.808086 4212 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:57:46.824553 4212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Generated new TSK 0
I20260621 06:57:46.825469 4212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:57:46.875854 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34549
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:47.205564 4225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:47.205915 4225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:47.206089 4225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:47.213186 4225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:47.213359 4225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:47.213505 4225 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:47.213639 4225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:57:47.223520 4225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:34549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:47.224946 4225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:47.226317 4225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:47.239549 4230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:47.239596 4233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:47.239497 4231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:47.240453 4225 server_base.cc:1034] running on GCE node
I20260621 06:57:47.241108 4225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:47.242063 4225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:47.243338 4225 hybrid_clock.cc:648] HybridClock initialized: now 1782025067243265 us; error 43 us; skew 500 ppm
I20260621 06:57:47.243665 4225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:47 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025067, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:47.247126 4225 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:57:47.248502 4225 webserver.cc:460] Webserver started at http://127.2.245.193:36179/ using document root <none> and password file <none>
I20260621 06:57:47.249246 4225 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:47.249341 4225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:47.249591 4225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:47.252542 4225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "3d9e8c48a7cb952fc0b18d98ab66be8a"
server_key_iv: "bf9c81f37558d557c4ab57b50000c0b7"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.253279 4225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "3d9e8c48a7cb952fc0b18d98ab66be8a"
server_key_iv: "bf9c81f37558d557c4ab57b50000c0b7"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.258509 4225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:57:47.262028 4240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:47.263492 4225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:57:47.263623 4225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "3d9e8c48a7cb952fc0b18d98ab66be8a"
server_key_iv: "bf9c81f37558d557c4ab57b50000c0b7"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.263789 4225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:47.279592 4225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:47.283713 4225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:47.283931 4225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:47.286712 4225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:47.286829 4225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:47.287014 4225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:47.287114 4225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Jun 21 06:57:47 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025067, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:47.310467 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.287449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:46947 (local address 127.2.245.254:34549)
0621 06:57:47.288064 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:47.288072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:47.289211 (+ 1139us) server_negotiation.cc:407] Connection header received
0621 06:57:47.291855 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:47.291864 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:47.291963 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:47.292055 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:47.294677 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.295974 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.297643 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.298067 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.298205 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.303391 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:47.303396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:47.303399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:47.303433 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:47.307028 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.308084 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.308092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.308095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.308222 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.308677 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.308683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.308686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.309003 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:47.309168 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:47.309854 (+ 686us) server_negotiation.cc:299] Negotiation successful
0621 06:57:47.310073 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":368,"threads_started":1}
I20260621 06:57:47.311717 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.288142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.193:46947)
0621 06:57:47.289031 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:47.289076 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:47.291450 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:47.292650 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:47.292670 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:47.293461 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:47.294465 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.294499 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.296141 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.296148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.297482 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.297495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.298327 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.298335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.298560 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.299412 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:47.299449 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:47.303089 (+ 3640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:47.307190 (+ 4101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.307198 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.307214 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.307939 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.308347 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.308353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.308357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.308558 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.309148 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:47.309156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:47.309504 (+ 348us) client_negotiation.cc:769] Sending connection context
0621 06:57:47.309777 (+ 273us) client_negotiation.cc:240] Negotiation successful
0621 06:57:47.310340 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":377,"threads_started":1}
I20260621 06:57:47.336606 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.318701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:49947 (local address 127.2.245.254:34549)
0621 06:57:47.319171 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:47.319180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:47.319963 (+ 783us) server_negotiation.cc:407] Connection header received
0621 06:57:47.320176 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:47.320181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:47.320269 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:47.320384 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:47.322317 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.323637 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.325638 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.326567 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.326730 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.329662 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:47.329673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:47.329681 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:47.329730 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:47.333402 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.334262 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.334269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.334274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.334449 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.335083 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.335088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.335091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.335408 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:47.335572 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:47.336003 (+ 431us) server_negotiation.cc:299] Negotiation successful
0621 06:57:47.336233 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313}
I20260621 06:57:47.336712 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.318923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.193:49947)
0621 06:57:47.319852 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:47.319870 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:47.320033 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:47.320591 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:47.320599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:47.321525 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:47.322091 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.322113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.323807 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.323815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.325461 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.325473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.326813 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.326821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.327058 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.327818 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:47.327848 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:47.329392 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:47.333564 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.333572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.333579 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.334126 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.334616 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.334628 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.334634 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.334901 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.335538 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:47.335546 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:47.335735 (+ 189us) client_negotiation.cc:769] Sending connection context
0621 06:57:47.335879 (+ 144us) client_negotiation.cc:240] Negotiation successful
0621 06:57:47.336224 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":518,"threads_started":1}
I20260621 06:57:47.346196 4225 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:33323
I20260621 06:57:47.346227 4360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:33323 every 8 connection(s)
I20260621 06:57:47.347524 4225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:57:47.349771 4361 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34549
I20260621 06:57:47.350008 4361 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:47.350716 4361 heartbeater.cc:511] Master 127.2.245.254:34549 requested a full tablet report, sending...
I20260621 06:57:47.352689 4154 ts_manager.cc:194] Registered new tserver with Master: 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323)
I20260621 06:57:47.354132 4154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:46947
I20260621 06:57:47.355792 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4225
I20260621 06:57:47.356001 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:57:47.356612 3031 external_mini_cluster.cc:1428] Setting key 17b4a6628de1bf05ea9ba7b2814c94a0
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:57:47.394678 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34549
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:47.714381 4365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:47.714761 4365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:47.714932 4365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:47.722178 4365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:47.722348 4365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:47.722517 4365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:47.722658 4365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:57:47.732611 4365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:34549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:47.733839 4365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:47.735236 4365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:47.747799 4370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:47.747828 4371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:47.748034 4373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:47.748716 4365 server_base.cc:1034] running on GCE node
I20260621 06:57:47.749342 4365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:47.750229 4365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:47.751480 4365 hybrid_clock.cc:648] HybridClock initialized: now 1782025067751415 us; error 72 us; skew 500 ppm
I20260621 06:57:47.751783 4365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:47 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025067, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:47.759720 4365 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:57:47.761301 4365 webserver.cc:460] Webserver started at http://127.2.245.194:40379/ using document root <none> and password file <none>
I20260621 06:57:47.762063 4365 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:47.762199 4365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:47.762589 4365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:47.765560 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "d2d79597b19641f1ada2157fece2f857"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "9efabb15fce49f8558b84dae1a51adbf"
server_key_iv: "2e3bd45a5d8223b88790f8386be74100"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.766352 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "d2d79597b19641f1ada2157fece2f857"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "9efabb15fce49f8558b84dae1a51adbf"
server_key_iv: "2e3bd45a5d8223b88790f8386be74100"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.771889 4365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260621 06:57:47.775548 4380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:47.776841 4365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:57:47.777015 4365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "d2d79597b19641f1ada2157fece2f857"
format_stamp: "Formatted at 2026-06-21 06:57:47 on dist-test-slave-9zdv"
server_key: "9efabb15fce49f8558b84dae1a51adbf"
server_key_iv: "2e3bd45a5d8223b88790f8386be74100"
server_key_version: "encryptionkey@0"
I20260621 06:57:47.777244 4365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:47.790912 4365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:47.794814 4365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:47.795146 4365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:47.797652 4365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:47.797783 4365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:47.797942 4365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:47.798024 4365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:57:47 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025067, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:47.822150 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.798544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:47179 (local address 127.2.245.254:34549)
0621 06:57:47.798754 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:47.798760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:47.800171 (+ 1411us) server_negotiation.cc:407] Connection header received
0621 06:57:47.802162 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:47.802168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:47.802294 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:47.802424 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:47.804620 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.805713 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.808398 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.809042 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.809184 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.814655 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:47.814662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:47.814667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:47.814711 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:47.818061 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.819080 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.819085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.819088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.819263 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.819846 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.819851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.819859 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.820134 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:47.820297 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:47.821439 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0621 06:57:47.821714 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 06:57:47.824546 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.799094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.194:47179)
0621 06:57:47.799954 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:47.800028 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:47.801875 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:47.802669 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:47.802677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:47.803450 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:47.804420 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.804448 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.805890 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.805902 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.808213 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.808231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.809311 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.809319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.809648 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.810514 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:47.810537 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:47.814346 (+ 3809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:47.818222 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.818231 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.818250 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.818893 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.819417 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.819423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.819429 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.819685 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.820275 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:47.820283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:47.820857 (+ 574us) client_negotiation.cc:769] Sending connection context
0621 06:57:47.822410 (+ 1553us) client_negotiation.cc:240] Negotiation successful
0621 06:57:47.823055 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":408,"threads_started":1}
I20260621 06:57:47.843521 4365 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:41091
I20260621 06:57:47.843524 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:41091 every 8 connection(s)
I20260621 06:57:47.844880 4365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:57:47.847060 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.831827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:42873 (local address 127.2.245.254:34549)
0621 06:57:47.832035 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:47.832043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:47.832945 (+ 902us) server_negotiation.cc:407] Connection header received
0621 06:57:47.833166 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:47.833170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:47.833237 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:47.833316 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:47.834746 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.835668 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.837756 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.838408 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.838547 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.840933 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:47.840938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:47.840941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:47.840971 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:47.843956 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.844629 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.844637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.844640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.844770 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:47.845189 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:47.845194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:47.845198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:47.845721 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:47.845916 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:47.846468 (+ 552us) server_negotiation.cc:299] Negotiation successful
0621 06:57:47.846731 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:57:47.847100 4500 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34549
I20260621 06:57:47.847208 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:47.831993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.194:42873)
0621 06:57:47.832796 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:47.832820 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:47.833029 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:47.833387 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:47.833394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:47.834116 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:47.834570 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.834587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.835804 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.835810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.837601 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:47.837613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:47.838615 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:47.838621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:47.838772 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:47.839581 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:47.839608 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:47.840755 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:47.844092 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.844098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.844102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.844499 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.844896 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:47.844901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:47.844905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:47.845067 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:47.845976 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:47.845981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:47.846218 (+ 237us) client_negotiation.cc:769] Sending connection context
0621 06:57:47.846370 (+ 152us) client_negotiation.cc:240] Negotiation successful
0621 06:57:47.846727 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":378,"threads_started":1}
I20260621 06:57:47.847338 4500 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:47.848055 4500 heartbeater.cc:511] Master 127.2.245.254:34549 requested a full tablet report, sending...
I20260621 06:57:47.849489 4154 ts_manager.cc:194] Registered new tserver with Master: d2d79597b19641f1ada2157fece2f857 (127.2.245.194:41091)
I20260621 06:57:47.850672 4154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:47179
I20260621 06:57:47.851665 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4365
I20260621 06:57:47.851850 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:57:47.852290 3031 external_mini_cluster.cc:1428] Setting key b4d0913fd6ceb5af72926784307b8795
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:57:47.889879 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34549
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:36835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:48.211335 4504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:48.211691 4504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:48.211863 4504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:48.219184 4504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:48.219331 4504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:48.219480 4504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:48.219615 4504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:57:48.229643 4504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:36835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:34549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:48.231042 4504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:48.232398 4504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:48.245155 4512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:48.245296 4509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:48.245158 4510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:48.246111 4504 server_base.cc:1034] running on GCE node
I20260621 06:57:48.246780 4504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:48.247759 4504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:48.248997 4504 hybrid_clock.cc:648] HybridClock initialized: now 1782025068248936 us; error 38 us; skew 500 ppm
I20260621 06:57:48.249338 4504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:48 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:48.252693 4504 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:57:48.254143 4504 webserver.cc:460] Webserver started at http://127.2.245.195:42805/ using document root <none> and password file <none>
I20260621 06:57:48.254803 4504 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:48.254943 4504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:48.255308 4504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:48.258097 4504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "fc031edafaac418d843791c3a8ac45b5"
format_stamp: "Formatted at 2026-06-21 06:57:48 on dist-test-slave-9zdv"
server_key: "0b7ea169688d64320f7b7f9ecf38f5a0"
server_key_iv: "97e709d80840d4ade9a733ee3a413be8"
server_key_version: "encryptionkey@0"
I20260621 06:57:48.258836 4504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "fc031edafaac418d843791c3a8ac45b5"
format_stamp: "Formatted at 2026-06-21 06:57:48 on dist-test-slave-9zdv"
server_key: "0b7ea169688d64320f7b7f9ecf38f5a0"
server_key_iv: "97e709d80840d4ade9a733ee3a413be8"
server_key_version: "encryptionkey@0"
I20260621 06:57:48.264081 4504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 06:57:48.268067 4519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:48.269627 4504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:57:48.269791 4504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "fc031edafaac418d843791c3a8ac45b5"
format_stamp: "Formatted at 2026-06-21 06:57:48 on dist-test-slave-9zdv"
server_key: "0b7ea169688d64320f7b7f9ecf38f5a0"
server_key_iv: "97e709d80840d4ade9a733ee3a413be8"
server_key_version: "encryptionkey@0"
I20260621 06:57:48.270009 4504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:48.298748 4504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:48.302845 4504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:48.303145 4504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:48.305855 4504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:48.305948 4504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:48.306088 4504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:48.306237 4504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 21 06:57:48 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:48.330771 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.306422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:46801 (local address 127.2.245.254:34549)
0621 06:57:48.306682 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.306690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.308116 (+ 1426us) server_negotiation.cc:407] Connection header received
0621 06:57:48.310272 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.310278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.310365 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.310526 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:48.312828 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.314009 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.316564 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.316995 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.317144 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.322529 (+ 5385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:48.322535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:48.322538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:48.322573 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:48.326775 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.328068 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.328075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.328080 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.328238 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.328889 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.328893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.328895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.329165 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:48.329328 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.330143 (+ 815us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.330401 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260621 06:57:48.331851 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.307150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.195:46801)
0621 06:57:48.307943 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.307994 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.309962 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.310772 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.310780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.311605 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:48.312550 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.312587 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.314176 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.314182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.316409 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.316423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.317258 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.317265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.317482 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.318295 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:48.318341 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:48.322222 (+ 3881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:48.327013 (+ 4791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.327027 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.327048 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.327880 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.328364 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.328370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.328379 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.328720 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.329300 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:48.329308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:48.329685 (+ 377us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.329983 (+ 298us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.330562 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":349,"threads_started":1}
I20260621 06:57:48.354156 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.338178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:46105 (local address 127.2.245.254:34549)
0621 06:57:48.338404 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.338413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.339143 (+ 730us) server_negotiation.cc:407] Connection header received
0621 06:57:48.339460 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.339469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.339560 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.339635 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:48.341644 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.342971 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.345604 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.346125 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.346277 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.348434 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:48.348439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:48.348442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:48.348472 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:48.351389 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.352189 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.352194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.352198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.352322 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.352696 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.352701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.352705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.352995 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:48.353157 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.353573 (+ 416us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.353792 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 06:57:48.354243 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.338181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.2.245.195:46105)
0621 06:57:48.339005 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.339032 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.339275 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.339889 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.339900 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.340712 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:48.341430 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.341450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.343185 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.343193 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.345426 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.345446 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.346339 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.346345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.346632 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.347324 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:48.347349 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:48.348232 (+ 883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:48.351535 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.351541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.351546 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.352061 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.352434 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.352440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.352444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.352585 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.353115 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:48.353122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:48.353303 (+ 181us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.353466 (+ 163us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.353801 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":377,"threads_started":1}
I20260621 06:57:48.356901 4361 heartbeater.cc:503] Master 127.2.245.254:34549 was elected leader, sending a full tablet report...
I20260621 06:57:48.370568 4504 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:44465
I20260621 06:57:48.370586 4638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:44465 every 8 connection(s)
I20260621 06:57:48.371872 4504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:57:48.374194 4639 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34549
I20260621 06:57:48.374421 4639 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:48.375192 4639 heartbeater.cc:511] Master 127.2.245.254:34549 requested a full tablet report, sending...
I20260621 06:57:48.376482 4154 ts_manager.cc:194] Registered new tserver with Master: fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:48.377291 4154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:46801
I20260621 06:57:48.378019 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4504
I20260621 06:57:48.378229 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:57:48.378647 3031 external_mini_cluster.cc:1428] Setting key 21548b4342a74e18255155b4e512df8a
I20260621 06:57:48.381474 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 06:57:48 dist-test-slave-9zdv krb5kdc[4109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 06:57:48 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:48.426934 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.412946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45648 (local address 127.2.245.254:34549)
0621 06:57:48.413135 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.413139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.413491 (+ 352us) server_negotiation.cc:407] Connection header received
0621 06:57:48.413668 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.413672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.413750 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.413845 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:48.415267 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.415983 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.417240 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.417649 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.417744 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.421821 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:48.421826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:48.421829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:48.421865 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:48.424548 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.425144 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.425148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.425150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.425271 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.425595 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.425598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.425600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.425849 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:48.426022 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.426351 (+ 329us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.426545 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 06:57:48.431548 4154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:48.438120 4154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:48.475194 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.450858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.2.245.194:41091)
0621 06:57:48.464759 (+ 13901us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.464773 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.465573 (+ 800us) server_negotiation.cc:407] Connection header received
0621 06:57:48.465713 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.465724 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.466087 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.466320 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:48.467809 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.469432 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.472170 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.473902 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.474060 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.474319 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.474713 (+ 394us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.474874 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13722,"thread_start_us":13596,"threads_started":1}
I20260621 06:57:48.475338 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.450497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:41091 (local address 127.0.0.1:43016)
0621 06:57:48.465220 (+ 14723us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.465244 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.465402 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.466894 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.466906 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.467010 (+ 104us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:48.467545 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.467562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.469586 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.469595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.471943 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.471969 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.474130 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.474138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.474463 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.474502 (+ 39us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.474643 (+ 141us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.474846 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14534,"mutex_wait_us":1,"thread_start_us":12389,"threads_started":1}
I20260621 06:57:48.477205 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.450223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:33323 (local address 127.0.0.1:52350)
0621 06:57:48.462324 (+ 12101us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.462386 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.462647 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.469638 (+ 6991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.469642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.469673 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:48.470217 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.470227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.471960 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.471965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.474152 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.474173 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.476443 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.476450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.476609 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.476639 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.476755 (+ 116us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.476894 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11714,"mutex_wait_us":1,"thread_start_us":18276,"threads_started":1}
I20260621 06:57:48.477303 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.450498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.2.245.193:33323)
0621 06:57:48.468903 (+ 18405us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.468913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.468952 (+ 39us) server_negotiation.cc:407] Connection header received
0621 06:57:48.469047 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.469055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.469273 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.469477 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:48.470393 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.471782 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.474354 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.476228 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.476367 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.476560 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.476820 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.476996 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17822,"spinlock_wait_cycles":789760,"thread_start_us":17833,"threads_started":1}
I20260621 06:57:48.479015 4430 tablet_service.cc:1467] Processing CreateTablet for tablet 60cd317f2ae847a184a2f429d6b54961 (DEFAULT_TABLE table=test-table [id=c44bfa14443643119d9860a1e8a960c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.480508 4430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cd317f2ae847a184a2f429d6b54961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.481304 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.464318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.2.245.195:44465)
0621 06:57:48.468297 (+ 3979us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.468309 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.470111 (+ 1802us) server_negotiation.cc:407] Connection header received
0621 06:57:48.470217 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.470224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.470497 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.470708 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:48.471416 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.472996 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.475958 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.480081 (+ 4123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.480222 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.480435 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.480807 (+ 372us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.480992 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3754,"thread_start_us":3528,"threads_started":1}
I20260621 06:57:48.480872 4291 tablet_service.cc:1467] Processing CreateTablet for tablet 60cd317f2ae847a184a2f429d6b54961 (DEFAULT_TABLE table=test-table [id=c44bfa14443643119d9860a1e8a960c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.481348 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.463916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:44465 (local address 127.0.0.1:54036)
0621 06:57:48.469880 (+ 5964us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.469903 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.470047 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.470729 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.470734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.470762 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:48.471189 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.471201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.473148 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.473153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.475379 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.475393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.480273 (+ 4880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.480280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.480521 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.480609 (+ 88us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.480693 (+ 84us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.480867 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4711,"mutex_wait_us":4,"thread_start_us":2631,"threads_started":1}
I20260621 06:57:48.482271 4291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cd317f2ae847a184a2f429d6b54961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.485177 4569 tablet_service.cc:1467] Processing CreateTablet for tablet 60cd317f2ae847a184a2f429d6b54961 (DEFAULT_TABLE table=test-table [id=c44bfa14443643119d9860a1e8a960c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.486542 4569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cd317f2ae847a184a2f429d6b54961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.493182 4658 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Bootstrap starting.
I20260621 06:57:48.496623 4658 tablet_bootstrap.cc:654] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.497822 4658 log.cc:824] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:48.503072 4659 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Bootstrap starting.
I20260621 06:57:48.503571 4661 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Bootstrap starting.
I20260621 06:57:48.506383 4659 tablet_bootstrap.cc:654] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.507018 4658 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: No bootstrap required, opened a new log
I20260621 06:57:48.507145 4661 tablet_bootstrap.cc:654] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.507319 4658 ts_tablet_manager.cc:1397] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260621 06:57:48.507736 4659 log.cc:824] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:48.508384 4661 log.cc:824] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:48.510782 4659 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: No bootstrap required, opened a new log
I20260621 06:57:48.511140 4659 ts_tablet_manager.cc:1397] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260621 06:57:48.518885 4659 raft_consensus.cc:348] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.519372 4659 raft_consensus.cc:374] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.519487 4659 raft_consensus.cc:729] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.520120 4659 consensus_queue.cc:260] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.520756 4658 raft_consensus.cc:348] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.521195 4658 raft_consensus.cc:374] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.521207 4500 heartbeater.cc:503] Master 127.2.245.254:34549 was elected leader, sending a full tablet report...
I20260621 06:57:48.521301 4658 raft_consensus.cc:729] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f444ed1b6d48cc84b4592ef3ffb4d9, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.521533 4661 tablet_bootstrap.cc:492] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: No bootstrap required, opened a new log
I20260621 06:57:48.521812 4661 ts_tablet_manager.cc:1397] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.012s
I20260621 06:57:48.521920 4658 consensus_queue.cc:260] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.523751 4658 ts_tablet_manager.cc:1428] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260621 06:57:48.524928 4659 ts_tablet_manager.cc:1428] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Time spent starting tablet: real 0.014s user 0.010s sys 0.000s
I20260621 06:57:48.530292 4664 raft_consensus.cc:482] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:48.530557 4664 raft_consensus.cc:504] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.530652 4661 raft_consensus.cc:348] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.531121 4661 raft_consensus.cc:374] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.531225 4661 raft_consensus.cc:729] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc031edafaac418d843791c3a8ac45b5, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.531744 4661 consensus_queue.cc:260] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.532390 4664 leader_election.cc:288] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:48.532912 4639 heartbeater.cc:503] Master 127.2.245.254:34549 was elected leader, sending a full tablet report...
I20260621 06:57:48.533421 4661 ts_tablet_manager.cc:1428] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260621 06:57:48.545442 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.532924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:40323 (local address 127.2.245.193:33323)
0621 06:57:48.533603 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.533609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.538922 (+ 5313us) server_negotiation.cc:407] Connection header received
0621 06:57:48.539169 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.539175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.539252 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.539348 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:48.539995 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.540996 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.543237 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.544451 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.544626 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.544702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.545069 (+ 367us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.545199 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522}
I20260621 06:57:48.545526 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.533980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:47461 (local address 127.2.245.195:44465)
0621 06:57:48.534215 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.534221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.539560 (+ 5339us) server_negotiation.cc:407] Connection header received
0621 06:57:48.539719 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.539722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.539771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.539836 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:48.540364 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.541371 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.543466 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.544670 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.544826 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.544873 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.545189 (+ 316us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.545303 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 06:57:48.545603 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.533218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:44465 (local address 127.2.245.194:47461)
0621 06:57:48.539430 (+ 6212us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.539449 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.539604 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.539874 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.539877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.539906 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:48.540222 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.540230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.541516 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.541532 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.543311 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.543325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.544865 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.544871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.545038 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.545078 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.545151 (+ 73us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.545244 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6074,"mutex_wait_us":21,"thread_start_us":5334,"threads_started":1}
I20260621 06:57:48.545830 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.532531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:33323 (local address 127.2.245.194:40323)
0621 06:57:48.538796 (+ 6265us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.538816 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.538932 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.539386 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.539391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.539424 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:48.539849 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.539860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.541164 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.541170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.543072 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.543085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.544671 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.544676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.544886 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.544923 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.545022 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.545177 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6161,"mutex_wait_us":28,"thread_start_us":6599,"threads_started":1}
I20260621 06:57:48.546831 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cd317f2ae847a184a2f429d6b54961" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5" is_pre_election: true
I20260621 06:57:48.546831 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cd317f2ae847a184a2f429d6b54961" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" is_pre_election: true
I20260621 06:57:48.547437 4316 raft_consensus.cc:2454] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:48.547439 4594 raft_consensus.cc:2454] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:48.548092 4384 leader_election.cc:302] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2d79597b19641f1ada2157fece2f857, fc031edafaac418d843791c3a8ac45b5; no voters:
I20260621 06:57:48.548422 4664 raft_consensus.cc:2781] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:48.548595 4664 raft_consensus.cc:482] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:48.548717 4664 raft_consensus.cc:3037] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.550863 4664 raft_consensus.cc:504] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.551591 4664 leader_election.cc:288] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Requested vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:48.551985 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cd317f2ae847a184a2f429d6b54961" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
I20260621 06:57:48.552038 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cd317f2ae847a184a2f429d6b54961" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5"
I20260621 06:57:48.552219 4594 raft_consensus.cc:3037] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.552219 4316 raft_consensus.cc:3037] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.555467 4316 raft_consensus.cc:2454] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
I20260621 06:57:48.555467 4594 raft_consensus.cc:2454] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
I20260621 06:57:48.555928 4384 leader_election.cc:302] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2d79597b19641f1ada2157fece2f857, fc031edafaac418d843791c3a8ac45b5; no voters:
I20260621 06:57:48.556231 4664 raft_consensus.cc:2781] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:48.556748 4664 raft_consensus.cc:686] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 1 LEADER]: Becoming Leader. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Running, Role: LEADER
I20260621 06:57:48.557299 4664 consensus_queue.cc:237] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.562091 4154 catalog_manager.cc:5495] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2d79597b19641f1ada2157fece2f857 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } health_report { overall_health: UNKNOWN } } }
I20260621 06:57:48.595062 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.589757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.2.245.194:41091)
0621 06:57:48.589942 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.589947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.590119 (+ 172us) server_negotiation.cc:407] Connection header received
0621 06:57:48.590343 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.590347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.590437 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.590600 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:48.590915 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.591704 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.592999 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.593400 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.593516 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.593947 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:48.594456 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:48.594562 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.594721 (+ 159us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.594784 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260621 06:57:48.597366 4501 tablet.cc:2369] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:57:48.599927 4362 tablet.cc:2369] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:57:48.603626 4316 raft_consensus.cc:1261] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Refusing update from remote peer d2d79597b19641f1ada2157fece2f857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:48.603626 4594 raft_consensus.cc:1261] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Refusing update from remote peer d2d79597b19641f1ada2157fece2f857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:48.604712 4669 consensus_queue.cc:1035] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:48.605058 4664 consensus_queue.cc:1035] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:48.639007 4154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:48.639292 4154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:48.642920 4154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Sending DeleteTablet for 3 replicas of tablet 60cd317f2ae847a184a2f429d6b54961
I20260621 06:57:48.644071 4291 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cd317f2ae847a184a2f429d6b54961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:48 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52350
I20260621 06:57:48.644531 4430 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cd317f2ae847a184a2f429d6b54961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:48 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:43016
I20260621 06:57:48.645200 4569 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cd317f2ae847a184a2f429d6b54961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:48 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:54036
I20260621 06:57:48.645761 4153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:48.646773 4153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:48.650027 4684 tablet_replica.cc:330] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: stopping tablet replica
I20260621 06:57:48.650496 4684 raft_consensus.cc:2229] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:48.651073 4684 raft_consensus.cc:2258] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:48.655628 4569 tablet_service.cc:1467] Processing CreateTablet for tablet 76b439f6fbb64bedad72059d48823e12 (DEFAULT_TABLE table=test-table [id=41bf30d4c5324b5291d5dbc98d36f685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.656122 4569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b439f6fbb64bedad72059d48823e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.656121 4429 tablet_service.cc:1467] Processing CreateTablet for tablet 76b439f6fbb64bedad72059d48823e12 (DEFAULT_TABLE table=test-table [id=41bf30d4c5324b5291d5dbc98d36f685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.656661 4429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b439f6fbb64bedad72059d48823e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.656665 4290 tablet_service.cc:1467] Processing CreateTablet for tablet 76b439f6fbb64bedad72059d48823e12 (DEFAULT_TABLE table=test-table [id=41bf30d4c5324b5291d5dbc98d36f685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:48.657213 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b439f6fbb64bedad72059d48823e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.663242 4661 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Bootstrap starting.
I20260621 06:57:48.664130 4658 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Bootstrap starting.
I20260621 06:57:48.664482 4682 tablet_replica.cc:330] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: stopping tablet replica
I20260621 06:57:48.664853 4661 tablet_bootstrap.cc:654] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.665102 4682 raft_consensus.cc:2229] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:48.665794 4658 tablet_bootstrap.cc:654] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.666139 4682 raft_consensus.cc:2258] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:48.667315 4659 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Bootstrap starting.
I20260621 06:57:48.669523 4659 tablet_bootstrap.cc:654] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.675458 4661 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: No bootstrap required, opened a new log
I20260621 06:57:48.675621 4661 ts_tablet_manager.cc:1397] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260621 06:57:48.676431 4661 raft_consensus.cc:348] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.676671 4661 raft_consensus.cc:374] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.676782 4661 raft_consensus.cc:729] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc031edafaac418d843791c3a8ac45b5, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.677142 4661 consensus_queue.cc:260] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.678560 4661 ts_tablet_manager.cc:1428] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260621 06:57:48.679113 4682 ts_tablet_manager.cc:1905] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:48.680688 4684 ts_tablet_manager.cc:1905] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:48.684151 4659 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: No bootstrap required, opened a new log
I20260621 06:57:48.684330 4659 ts_tablet_manager.cc:1397] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260621 06:57:48.685527 4659 raft_consensus.cc:348] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.685843 4659 raft_consensus.cc:374] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.685962 4659 raft_consensus.cc:729] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.686362 4659 consensus_queue.cc:260] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.691998 4684 ts_tablet_manager.cc:1918] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:57:48.692214 4684 log.cc:1196] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/60cd317f2ae847a184a2f429d6b54961
I20260621 06:57:48.692852 4684 ts_tablet_manager.cc:1939] T 60cd317f2ae847a184a2f429d6b54961 P fc031edafaac418d843791c3a8ac45b5: Deleting consensus metadata
I20260621 06:57:48.693120 4682 ts_tablet_manager.cc:1918] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:57:48.693310 4682 log.cc:1196] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/60cd317f2ae847a184a2f429d6b54961
I20260621 06:57:48.693846 4682 ts_tablet_manager.cc:1939] T 60cd317f2ae847a184a2f429d6b54961 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting consensus metadata
I20260621 06:57:48.695168 4141 catalog_manager.cc:4841] TS fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465): tablet 60cd317f2ae847a184a2f429d6b54961 (table test-table [id=c44bfa14443643119d9860a1e8a960c3]) successfully deleted
I20260621 06:57:48.696193 4139 catalog_manager.cc:4841] TS 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323): tablet 60cd317f2ae847a184a2f429d6b54961 (table test-table [id=c44bfa14443643119d9860a1e8a960c3]) successfully deleted
I20260621 06:57:48.696589 4659 ts_tablet_manager.cc:1428] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260621 06:57:48.697278 4658 tablet_bootstrap.cc:492] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: No bootstrap required, opened a new log
I20260621 06:57:48.697446 4658 ts_tablet_manager.cc:1397] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.026s
I20260621 06:57:48.697896 4683 tablet_replica.cc:330] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: stopping tablet replica
I20260621 06:57:48.698374 4658 raft_consensus.cc:348] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.698619 4658 raft_consensus.cc:374] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.698721 4658 raft_consensus.cc:729] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f444ed1b6d48cc84b4592ef3ffb4d9, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.699081 4658 consensus_queue.cc:260] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.699769 4658 ts_tablet_manager.cc:1428] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260621 06:57:48.700278 4683 raft_consensus.cc:2229] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:48.701056 4683 raft_consensus.cc:2258] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:48.703166 4683 ts_tablet_manager.cc:1905] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:48.708462 4683 ts_tablet_manager.cc:1918] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:57:48.708653 4683 log.cc:1196] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/60cd317f2ae847a184a2f429d6b54961
I20260621 06:57:48.709142 4683 ts_tablet_manager.cc:1939] T 60cd317f2ae847a184a2f429d6b54961 P d2d79597b19641f1ada2157fece2f857: Deleting consensus metadata
I20260621 06:57:48.710791 4141 catalog_manager.cc:4841] TS d2d79597b19641f1ada2157fece2f857 (127.2.245.194:41091): tablet 60cd317f2ae847a184a2f429d6b54961 (table test-table [id=c44bfa14443643119d9860a1e8a960c3]) successfully deleted
I20260621 06:57:48.721606 4664 raft_consensus.cc:482] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:48.721796 4664 raft_consensus.cc:504] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.722523 4664 leader_election.cc:288] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:48.723009 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b439f6fbb64bedad72059d48823e12" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" is_pre_election: true
I20260621 06:57:48.723248 4316 raft_consensus.cc:2454] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:48.723701 4382 leader_election.cc:302] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f444ed1b6d48cc84b4592ef3ffb4d9, d2d79597b19641f1ada2157fece2f857; no voters:
I20260621 06:57:48.723258 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b439f6fbb64bedad72059d48823e12" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5" is_pre_election: true
I20260621 06:57:48.724789 4594 raft_consensus.cc:2454] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:48.725528 4664 raft_consensus.cc:2781] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:48.725656 4664 raft_consensus.cc:482] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:48.725775 4664 raft_consensus.cc:3037] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.729403 4664 raft_consensus.cc:504] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.730810 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b439f6fbb64bedad72059d48823e12" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
I20260621 06:57:48.731034 4316 raft_consensus.cc:3037] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.734360 4316 raft_consensus.cc:2454] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
I20260621 06:57:48.734767 4382 leader_election.cc:302] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f444ed1b6d48cc84b4592ef3ffb4d9, d2d79597b19641f1ada2157fece2f857; no voters:
I20260621 06:57:48.735056 4677 raft_consensus.cc:2781] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:48.735231 4677 raft_consensus.cc:686] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 1 LEADER]: Becoming Leader. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Running, Role: LEADER
I20260621 06:57:48.735688 4677 consensus_queue.cc:237] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.738739 4664 leader_election.cc:288] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Requested vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:48.739393 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b439f6fbb64bedad72059d48823e12" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5"
I20260621 06:57:48.739598 4594 raft_consensus.cc:3037] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:48.739735 4153 catalog_manager.cc:5495] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2d79597b19641f1ada2157fece2f857 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } health_report { overall_health: UNKNOWN } } }
I20260621 06:57:48.742568 4594 raft_consensus.cc:2454] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
Jun 21 06:57:48 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:48.794340 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.777950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34549 (local address 127.0.0.1:45662)
0621 06:57:48.778556 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:48.778576 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:48.778745 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:48.779242 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:48.779246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:48.780165 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:48.780563 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.780576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.782171 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.782177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.783442 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:48.783457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.784400 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.784405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:48.784570 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.785302 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:48.785323 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:48.788199 (+ 2876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:48.791946 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.791956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.791962 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.792355 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.792765 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:48.792768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:48.792770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:48.792960 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:48.793543 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:48.793549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:48.793696 (+ 147us) client_negotiation.cc:769] Sending connection context
0621 06:57:48.793838 (+ 142us) client_negotiation.cc:240] Negotiation successful
0621 06:57:48.794024 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":334,"threads_started":1}
I20260621 06:57:48.794626 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:48.777884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.2.245.254:34549)
0621 06:57:48.778076 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:48.778087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:48.778649 (+ 562us) server_negotiation.cc:407] Connection header received
0621 06:57:48.778865 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:48.778870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:48.778980 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:48.779058 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:48.780791 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.782039 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.783558 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:48.784243 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:48.784366 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:48.788397 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:48.788406 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:48.788411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:48.788453 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:48.791774 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.792474 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.792479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.792483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.792655 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:48.793070 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:48.793075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:48.793079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:48.793422 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:48.793582 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:48.793973 (+ 391us) server_negotiation.cc:299] Negotiation successful
0621 06:57:48.794202 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 06:57:48.799546 4153 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:45662:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 06:57:48.800611 4153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:48.807778 4290 tablet_service.cc:1467] Processing CreateTablet for tablet 85804459c5ca45f1a1d09a6d114ab634 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c7e1d5dad7472f8b4ec61efbbda103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:48.808050 4430 tablet_service.cc:1467] Processing CreateTablet for tablet 85804459c5ca45f1a1d09a6d114ab634 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c7e1d5dad7472f8b4ec61efbbda103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:48.808343 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85804459c5ca45f1a1d09a6d114ab634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.808262 4569 tablet_service.cc:1467] Processing CreateTablet for tablet 85804459c5ca45f1a1d09a6d114ab634 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c7e1d5dad7472f8b4ec61efbbda103]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:48.808595 4430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85804459c5ca45f1a1d09a6d114ab634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.808794 4569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85804459c5ca45f1a1d09a6d114ab634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:48.815866 4659 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: Bootstrap starting.
I20260621 06:57:48.815850 4658 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Bootstrap starting.
I20260621 06:57:48.817523 4658 tablet_bootstrap.cc:654] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.817529 4659 tablet_bootstrap.cc:654] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.817741 4661 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: Bootstrap starting.
I20260621 06:57:48.819382 4661 tablet_bootstrap.cc:654] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:48.828709 4658 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: No bootstrap required, opened a new log
I20260621 06:57:48.829031 4658 ts_tablet_manager.cc:1397] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.008s
I20260621 06:57:48.829324 4661 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: No bootstrap required, opened a new log
I20260621 06:57:48.829563 4661 ts_tablet_manager.cc:1397] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.007s
I20260621 06:57:48.830073 4658 raft_consensus.cc:348] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.830394 4658 raft_consensus.cc:374] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.830572 4658 raft_consensus.cc:729] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f444ed1b6d48cc84b4592ef3ffb4d9, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.830446 4661 raft_consensus.cc:348] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.830732 4659 tablet_bootstrap.cc:492] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: No bootstrap required, opened a new log
I20260621 06:57:48.830734 4661 raft_consensus.cc:374] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.830806 4661 raft_consensus.cc:729] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc031edafaac418d843791c3a8ac45b5, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.830907 4659 ts_tablet_manager.cc:1397] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.010s
I20260621 06:57:48.831099 4661 consensus_queue.cc:260] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.831131 4658 consensus_queue.cc:260] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.831553 4666 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:48.831863 4666 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:48.831708 4665 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:48.832053 4665 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:48.832604 4658 ts_tablet_manager.cc:1428] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260621 06:57:48.832516 4659 raft_consensus.cc:348] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.832770 4659 raft_consensus.cc:374] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:48.832866 4659 raft_consensus.cc:729] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Initialized, Role: FOLLOWER
I20260621 06:57:48.833125 4659 consensus_queue.cc:260] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:48.833777 4659 ts_tablet_manager.cc:1428] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260621 06:57:48.833588 4664 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:48.833914 4664 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:48.835377 4661 ts_tablet_manager.cc:1428] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
W20260621 06:57:48.874598 4640 tablet.cc:2369] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:57:49.002185 4664 raft_consensus.cc:482] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:49.002429 4664 raft_consensus.cc:504] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:49.003216 4664 leader_election.cc:288] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:49.003799 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85804459c5ca45f1a1d09a6d114ab634" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" is_pre_election: true
I20260621 06:57:49.003865 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85804459c5ca45f1a1d09a6d114ab634" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5" is_pre_election: true
I20260621 06:57:49.004104 4594 raft_consensus.cc:2454] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:49.004098 4316 raft_consensus.cc:2454] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 0.
I20260621 06:57:49.004585 4382 leader_election.cc:302] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f444ed1b6d48cc84b4592ef3ffb4d9, d2d79597b19641f1ada2157fece2f857; no voters:
I20260621 06:57:49.004911 4664 raft_consensus.cc:2781] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:49.005067 4664 raft_consensus.cc:482] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:49.005192 4664 raft_consensus.cc:3037] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:49.007815 4664 raft_consensus.cc:504] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:49.008373 4664 leader_election.cc:288] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Requested vote from peers 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323), fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465)
I20260621 06:57:49.008793 4316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85804459c5ca45f1a1d09a6d114ab634" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9"
I20260621 06:57:49.008926 4594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85804459c5ca45f1a1d09a6d114ab634" candidate_uuid: "d2d79597b19641f1ada2157fece2f857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc031edafaac418d843791c3a8ac45b5"
I20260621 06:57:49.009034 4316 raft_consensus.cc:3037] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:49.009181 4594 raft_consensus.cc:3037] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:49.011991 4594 raft_consensus.cc:2454] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
I20260621 06:57:49.011991 4316 raft_consensus.cc:2454] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d79597b19641f1ada2157fece2f857 in term 1.
I20260621 06:57:49.012439 4384 leader_election.cc:302] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2d79597b19641f1ada2157fece2f857, fc031edafaac418d843791c3a8ac45b5; no voters:
I20260621 06:57:49.012769 4664 raft_consensus.cc:2781] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:49.012934 4664 raft_consensus.cc:686] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [term 1 LEADER]: Becoming Leader. State: Replica: d2d79597b19641f1ada2157fece2f857, State: Running, Role: LEADER
I20260621 06:57:49.013206 4664 consensus_queue.cc:237] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } }
I20260621 06:57:49.015003 4677 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: TxnStatusTablet state changed. Reason: New leader d2d79597b19641f1ada2157fece2f857. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.015491 4677 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:49.016130 4699 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 06:57:49.016369 4699 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 06:57:49.016726 4154 catalog_manager.cc:5495] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2d79597b19641f1ada2157fece2f857 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } health_report { overall_health: UNKNOWN } } }
I20260621 06:57:49.066715 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.060765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:41091 (local address 127.0.0.1:43042)
0621 06:57:49.060991 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:49.061014 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:49.061174 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:49.061844 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:49.061851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:49.061869 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:49.062031 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:49.062038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.063546 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.063550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:49.064534 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:49.064546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.065532 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.065536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:49.065683 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.065739 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:49.066230 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:49.066244 (+ 14us) client_negotiation.cc:769] Sending connection context
0621 06:57:49.066302 (+ 58us) client_negotiation.cc:240] Negotiation successful
0621 06:57:49.066412 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260621 06:57:49.066756 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.061188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.2.245.194:41091)
0621 06:57:49.061414 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.061420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.061442 (+ 22us) server_negotiation.cc:407] Connection header received
0621 06:57:49.061516 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.061521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.061635 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.061749 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:49.062178 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.063396 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.064689 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.065365 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.065513 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.065954 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:49.066120 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:49.066225 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.066358 (+ 133us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.066447 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260621 06:57:49.071072 4316 raft_consensus.cc:1261] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Refusing update from remote peer d2d79597b19641f1ada2157fece2f857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:49.071069 4594 raft_consensus.cc:1261] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Refusing update from remote peer d2d79597b19641f1ada2157fece2f857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:49.071707 4677 consensus_queue.cc:1035] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:49.071998 4664 consensus_queue.cc:1035] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:49.080046 4665 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: TxnStatusTablet state changed. Reason: New leader d2d79597b19641f1ada2157fece2f857. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.080404 4665 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:49.081836 4666 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: TxnStatusTablet state changed. Reason: New leader d2d79597b19641f1ada2157fece2f857. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.082089 4666 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:49.083371 4664 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.083671 4664 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:49.085409 4669 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.085773 4669 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P d2d79597b19641f1ada2157fece2f857: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:49.085968 4672 mvcc.cc:205] Tried to move back new op lower bound from 7299174682908213248 to 7299174682680479744. Current Snapshot: MvccSnapshot[applied={T|T < 7299174682908213248}]
I20260621 06:57:49.086126 4665 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.086349 4665 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P 20f444ed1b6d48cc84b4592ef3ffb4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:49.087808 4666 tablet_replica.cc:439] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2d79597b19641f1ada2157fece2f857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 } } peers { permanent_uuid: "d2d79597b19641f1ada2157fece2f857" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41091 } } peers { permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 } } }
I20260621 06:57:49.088044 4666 tablet_replica.cc:442] T 85804459c5ca45f1a1d09a6d114ab634 P fc031edafaac418d843791c3a8ac45b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:49.111785 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.105312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59047 (local address 127.2.245.194:41091)
0621 06:57:49.105505 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.105513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.106146 (+ 633us) server_negotiation.cc:407] Connection header received
0621 06:57:49.106325 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.106329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.106391 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.106464 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:49.107034 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.107887 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.109874 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.110434 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.110665 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.111134 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:49.111236 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:49.111309 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.111452 (+ 143us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.111511 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":60}
I20260621 06:57:49.111989 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.105330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:41091 (local address 127.2.245.194:59047)
0621 06:57:49.105988 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:49.106012 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:49.106171 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:49.106503 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:49.106508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:49.106519 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:49.106848 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:49.106859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.108006 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.108011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:49.109741 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:49.109757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.110677 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.110684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:49.110879 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.110967 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:49.111335 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:49.111346 (+ 11us) client_negotiation.cc:769] Sending connection context
0621 06:57:49.111429 (+ 83us) client_negotiation.cc:240] Negotiation successful
0621 06:57:49.111566 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":314,"threads_started":1}
I20260621 06:57:49.121147 4669 consensus_queue.cc:1035] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f444ed1b6d48cc84b4592ef3ffb4d9" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:49.121827 4594 raft_consensus.cc:1261] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Refusing update from remote peer d2d79597b19641f1ada2157fece2f857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:49.123122 4703 consensus_queue.cc:1035] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc031edafaac418d843791c3a8ac45b5" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:49.187160 4152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:49.187462 4152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:49.190739 4152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89cc0189b594571b42a01d8baeee6eb: Sending DeleteTablet for 3 replicas of tablet 76b439f6fbb64bedad72059d48823e12
I20260621 06:57:49.191594 4290 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b439f6fbb64bedad72059d48823e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:49 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52350
I20260621 06:57:49.191812 4569 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b439f6fbb64bedad72059d48823e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:49 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:54036
I20260621 06:57:49.191900 4682 tablet_replica.cc:330] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: stopping tablet replica
I20260621 06:57:49.191989 4430 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b439f6fbb64bedad72059d48823e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:49 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:43016
I20260621 06:57:49.192061 4684 tablet_replica.cc:330] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: stopping tablet replica
I20260621 06:57:49.192090 4682 raft_consensus.cc:2229] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:49.192250 4684 raft_consensus.cc:2229] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:49.192273 4682 raft_consensus.cc:2258] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:49.192430 4684 raft_consensus.cc:2258] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:49.192509 4683 tablet_replica.cc:330] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: stopping tablet replica
I20260621 06:57:49.192704 4683 raft_consensus.cc:2229] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:49.193063 4683 raft_consensus.cc:2258] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:49.193389 4684 ts_tablet_manager.cc:1905] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:49.193427 4682 ts_tablet_manager.cc:1905] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:49.200048 4683 ts_tablet_manager.cc:1905] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:49.200798 4682 ts_tablet_manager.cc:1918] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:49.200971 4682 log.cc:1196] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/76b439f6fbb64bedad72059d48823e12
I20260621 06:57:49.201400 4682 ts_tablet_manager.cc:1939] T 76b439f6fbb64bedad72059d48823e12 P 20f444ed1b6d48cc84b4592ef3ffb4d9: Deleting consensus metadata
I20260621 06:57:49.202574 4139 catalog_manager.cc:4841] TS 20f444ed1b6d48cc84b4592ef3ffb4d9 (127.2.245.193:33323): tablet 76b439f6fbb64bedad72059d48823e12 (table test-table [id=41bf30d4c5324b5291d5dbc98d36f685]) successfully deleted
I20260621 06:57:49.204535 4684 ts_tablet_manager.cc:1918] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:49.204715 4684 log.cc:1196] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/76b439f6fbb64bedad72059d48823e12
I20260621 06:57:49.205185 4684 ts_tablet_manager.cc:1939] T 76b439f6fbb64bedad72059d48823e12 P fc031edafaac418d843791c3a8ac45b5: Deleting consensus metadata
I20260621 06:57:49.206359 4141 catalog_manager.cc:4841] TS fc031edafaac418d843791c3a8ac45b5 (127.2.245.195:44465): tablet 76b439f6fbb64bedad72059d48823e12 (table test-table [id=41bf30d4c5324b5291d5dbc98d36f685]) successfully deleted
I20260621 06:57:49.207835 4683 ts_tablet_manager.cc:1918] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:49.208560 4683 log.cc:1196] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/76b439f6fbb64bedad72059d48823e12
I20260621 06:57:49.209090 4683 ts_tablet_manager.cc:1939] T 76b439f6fbb64bedad72059d48823e12 P d2d79597b19641f1ada2157fece2f857: Deleting consensus metadata
I20260621 06:57:49.210322 4141 catalog_manager.cc:4841] TS d2d79597b19641f1ada2157fece2f857 (127.2.245.194:41091): tablet 76b439f6fbb64bedad72059d48823e12 (table test-table [id=41bf30d4c5324b5291d5dbc98d36f685]) successfully deleted
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:49.225566 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.208019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52358 (local address 127.2.245.193:33323)
0621 06:57:49.208612 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.208618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.209201 (+ 583us) server_negotiation.cc:407] Connection header received
0621 06:57:49.209483 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.209488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.209569 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.209719 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:49.211680 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.212454 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.213770 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.214206 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.214346 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.218430 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:49.218452 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:49.218488 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:49.218554 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:49.222477 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.223351 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.223361 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.223364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.223527 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.223895 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.223901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.223905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.224357 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:49.224649 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.224877 (+ 228us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.225145 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
W20260621 06:57:49.226461 4356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52358
I20260621 06:57:49.243076 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.229509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.2.245.254:34549)
0621 06:57:49.229724 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.229730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.229916 (+ 186us) server_negotiation.cc:407] Connection header received
0621 06:57:49.230059 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.230066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.230157 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.230266 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:49.231567 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.232688 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.234022 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.234719 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.234838 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.237183 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:49.237189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:49.237192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:49.237225 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:49.240304 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.241054 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.241059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.241062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.241192 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.241572 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.241577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.241585 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.241862 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:49.242180 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.242467 (+ 287us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.242697 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260621 06:57:49.243662 4152 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678
I20260621 06:57:49.245615 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4225
I20260621 06:57:49.259951 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.246404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.2.245.193:33323)
0621 06:57:49.247150 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.247158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.247180 (+ 22us) server_negotiation.cc:407] Connection header received
0621 06:57:49.247236 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.247241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.247319 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.247421 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:49.248535 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.249265 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.250564 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.251302 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.251436 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.253832 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:49.253838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:49.253843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:49.253884 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:49.257309 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.257965 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.257971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.257975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.258155 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.258469 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.258474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.258497 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.258851 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:49.259123 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.259382 (+ 259us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.259609 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620}
W20260621 06:57:49.260620 4356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52368
W20260621 06:57:49.261220 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4225: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:49.261421 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4225
I20260621 06:57:49.279254 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4365
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:57:49.296974 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.280084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.2.245.194:41091)
0621 06:57:49.280287 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.280292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.280308 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:57:49.280520 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.280524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.280586 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.280748 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:49.282125 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.283300 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.284495 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.285229 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.285367 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.289615 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:49.289636 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:49.289659 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:49.289712 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:49.293918 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.294633 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.294643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.294647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.294807 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.295256 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.295261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.295264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.295780 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:49.296107 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.296318 (+ 211us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.296575 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260621 06:57:49.297745 4495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43058
W20260621 06:57:49.298441 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4365: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:49.298627 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4365
I20260621 06:57:49.320878 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4504
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025068, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:57:49.335876 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:49.321689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.2.245.195:44465)
0621 06:57:49.322167 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:49.322172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:49.322188 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:57:49.322242 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:49.322246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:49.322311 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:49.322408 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:49.323680 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.324430 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.325675 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:49.326083 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:49.326189 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:49.330353 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:49.330368 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:49.330384 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:49.330424 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:49.333174 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.333809 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.333816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.333819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.334006 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:49.334336 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:49.334340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:49.334342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:49.334708 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:49.334921 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:49.335288 (+ 367us) server_negotiation.cc:299] Negotiation successful
0621 06:57:49.335453 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":269,"threads_started":1}
W20260621 06:57:49.336503 4634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54042
W20260621 06:57:49.337090 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4504: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:49.337256 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4504
I20260621 06:57:49.353871 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4122
I20260621 06:57:49.355103 4194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:49.472064 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4122
2026-06-21T06:57:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4441 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4721](info): set up 2 sockets
Jun 21 06:57:49 dist-test-slave-9zdv krb5kdc[4721](info): commencing operation
krb5kdc: starting...
W20260621 06:57:50.559389 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:57:50 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:57:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:57:50.746403 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38275
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:51.064042 4734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:51.064332 4734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:51.064520 4734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:51.071869 4734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:57:51.071983 4734 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 06:57:51.072055 4734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:51.072151 4734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:51.072247 4734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:57:51.072358 4734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:57:51.082635 4734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38275
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:51.084023 4734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:51.085373 4734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:51.096947 4740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:51.097007 4742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:51.097443 4739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:51.098300 4734 server_base.cc:1034] running on GCE node
I20260621 06:57:51.099131 4734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:51.100715 4734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:51.102006 4734 hybrid_clock.cc:648] HybridClock initialized: now 1782025071101929 us; error 52 us; skew 500 ppm
I20260621 06:57:51.102362 4734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:51 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025071, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:51.105629 4734 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:57:51.107249 4734 webserver.cc:460] Webserver started at http://127.2.245.254:46643/ using document root <none> and password file <none>
I20260621 06:57:51.108001 4734 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:51.108137 4734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:51.108453 4734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:51.111531 4734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "3d9b4a4a875b407fadaee2c50f5f1110"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "d6c3b7c1ba878145b89c81c37a188e7e"
server_key_iv: "53babc0229b665906de1b9f250bde60a"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.112309 4734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "3d9b4a4a875b407fadaee2c50f5f1110"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "d6c3b7c1ba878145b89c81c37a188e7e"
server_key_iv: "53babc0229b665906de1b9f250bde60a"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.117652 4734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:57:51.121150 4749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:51.122460 4734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:57:51.122655 4734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "3d9b4a4a875b407fadaee2c50f5f1110"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "d6c3b7c1ba878145b89c81c37a188e7e"
server_key_iv: "53babc0229b665906de1b9f250bde60a"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.122876 4734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:51.135632 4734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:51.139776 4734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:51.140079 4734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:51.161928 4734 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:38275
I20260621 06:57:51.161947 4810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:38275 every 8 connection(s)
I20260621 06:57:51.163728 4734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:57:51.164135 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4734
I20260621 06:57:51.164366 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:57:51.165205 3031 external_mini_cluster.cc:1428] Setting key fce99deb90adab6f92b6abe95032a454
I20260621 06:57:51.168489 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 06:57:51 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:51.182065 4811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Bootstrap starting.
I20260621 06:57:51.186219 4811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:51.187611 4811 log.cc:824] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:51.189664 4814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:51.167332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.2.245.254:38275)
0621 06:57:51.168126 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:51.168137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:51.168177 (+ 40us) server_negotiation.cc:407] Connection header received
0621 06:57:51.169858 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:51.169880 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:51.170279 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:51.170592 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:51.172415 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.173761 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.175508 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.176164 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.176307 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:51.181083 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:51.181106 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:51.181125 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:51.181177 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:51.185107 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.185811 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.185819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.185826 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.186027 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.186462 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.186467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.186470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.187230 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:51.187556 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:51.187892 (+ 336us) server_negotiation.cc:299] Negotiation successful
0621 06:57:51.188377 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":392,"threads_started":1}
I20260621 06:57:51.190835 4811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: No bootstrap required, opened a new log
I20260621 06:57:51.197889 4811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } }
I20260621 06:57:51.198282 4811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:51.198388 4811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9b4a4a875b407fadaee2c50f5f1110, State: Initialized, Role: FOLLOWER
I20260621 06:57:51.199038 4811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } }
I20260621 06:57:51.199365 4811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:57:51.199522 4811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:57:51.199741 4811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:51.201826 4811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } }
I20260621 06:57:51.202275 4811 leader_election.cc:302] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d9b4a4a875b407fadaee2c50f5f1110; no voters:
I20260621 06:57:51.202826 4811 leader_election.cc:288] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:57:51.203037 4816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:51.203723 4816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [term 1 LEADER]: Becoming Leader. State: Replica: 3d9b4a4a875b407fadaee2c50f5f1110, State: Running, Role: LEADER
I20260621 06:57:51.204303 4816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } }
I20260621 06:57:51.205076 4811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:57:51.206804 4818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d9b4a4a875b407fadaee2c50f5f1110. Latest consensus state: current_term: 1 leader_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } } }
I20260621 06:57:51.207247 4818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:51.207684 4817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9b4a4a875b407fadaee2c50f5f1110" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38275 } } }
I20260621 06:57:51.208041 4817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110 [sys.catalog]: This master's current role is: LEADER
I20260621 06:57:51.209453 4821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:57:51.214828 4821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:57:51.224401 4821 catalog_manager.cc:1347] Generated new cluster ID: 96f1a062dc9d4af1bcef485a3a140ddf
I20260621 06:57:51.224550 4821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:57:51.233187 4821 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:57:51.235044 4821 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:57:51.242576 4821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Generated new TSK 0
I20260621 06:57:51.243623 4821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:57:51.294405 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38275
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:51.620708 4837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:51.620985 4837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:51.621213 4837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:51.628410 4837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:51.628548 4837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:51.628700 4837 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 06:57:51.628780 4837 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:51.628931 4837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:57:51.638695 4837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:51.640038 4837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:51.641400 4837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:51.654822 4843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:51.654901 4845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:51.654850 4842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:51.656023 4837 server_base.cc:1034] running on GCE node
I20260621 06:57:51.656651 4837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:51.657536 4837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:51.658735 4837 hybrid_clock.cc:648] HybridClock initialized: now 1782025071658687 us; error 39 us; skew 500 ppm
I20260621 06:57:51.659116 4837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:51 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025071, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:51.662252 4837 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:57:51.663818 4837 webserver.cc:460] Webserver started at http://127.2.245.193:46009/ using document root <none> and password file <none>
I20260621 06:57:51.664529 4837 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:51.664677 4837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:51.665009 4837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:51.668015 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "d379eacb6d6e45acb1d331b935a9f9be"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "5ac53f8bdff2dad1854cd7facd36aff4"
server_key_iv: "9248e35b08f5ddd7d1692a3bab86f6c8"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.668757 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "d379eacb6d6e45acb1d331b935a9f9be"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "5ac53f8bdff2dad1854cd7facd36aff4"
server_key_iv: "9248e35b08f5ddd7d1692a3bab86f6c8"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.674381 4837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:57:51.677979 4852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:51.679440 4837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:57:51.679620 4837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "d379eacb6d6e45acb1d331b935a9f9be"
format_stamp: "Formatted at 2026-06-21 06:57:51 on dist-test-slave-9zdv"
server_key: "5ac53f8bdff2dad1854cd7facd36aff4"
server_key_iv: "9248e35b08f5ddd7d1692a3bab86f6c8"
server_key_version: "encryptionkey@0"
I20260621 06:57:51.679826 4837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:51.700093 4837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:51.704139 4837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:51.704437 4837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:51.707477 4837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:51.707624 4837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:51.707769 4837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:51.707862 4837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:57:51 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025071, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:51.732671 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:51.707941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:40003 (local address 127.2.245.254:38275)
0621 06:57:51.708740 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:51.708747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:51.709850 (+ 1103us) server_negotiation.cc:407] Connection header received
0621 06:57:51.713041 (+ 3191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:51.713045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:51.713136 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:51.713229 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:51.715962 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.716819 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.719653 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.720080 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.720222 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:51.724597 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:51.724606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:51.724613 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:51.724672 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:51.728630 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.729590 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.729596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.729600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.729756 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.730347 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.730353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.730357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.730636 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:51.730805 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:51.731780 (+ 975us) server_negotiation.cc:299] Negotiation successful
0621 06:57:51.732099 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":543,"threads_started":1}
I20260621 06:57:51.733348 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:51.708635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.193:40003)
0621 06:57:51.709644 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:51.709714 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:51.712676 (+ 2962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:51.713586 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:51.713601 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:51.714657 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:51.715755 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:51.715788 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.716982 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.716987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:51.719476 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:51.719488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.720323 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.720332 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:51.720578 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:51.721325 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:51.721346 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:51.724280 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:51.728793 (+ 4513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:51.728806 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:51.728829 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:51.729431 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:51.729906 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:51.729912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:51.729917 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:51.730201 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:51.730766 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:51.730774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:51.731258 (+ 484us) client_negotiation.cc:769] Sending connection context
0621 06:57:51.731635 (+ 377us) client_negotiation.cc:240] Negotiation successful
0621 06:57:51.732068 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":399,"threads_started":1}
I20260621 06:57:51.755344 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:51.740190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:50361 (local address 127.2.245.254:38275)
0621 06:57:51.740366 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:51.740373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:51.741155 (+ 782us) server_negotiation.cc:407] Connection header received
0621 06:57:51.741342 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:51.741346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:51.741409 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:51.741519 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:51.742579 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.743540 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.745296 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.745907 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.746045 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:51.748509 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:51.748517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:51.748524 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:51.748580 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:51.752111 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.753002 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.753007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.753010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.753181 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:51.753720 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:51.753728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:51.753733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:51.754094 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:51.754267 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:51.754711 (+ 444us) server_negotiation.cc:299] Negotiation successful
0621 06:57:51.754964 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 06:57:51.755532 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:51.740301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.193:50361)
0621 06:57:51.741025 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:51.741054 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:51.741223 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:51.741545 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:51.741550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:51.742142 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:51.742436 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:51.742444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.743692 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.743696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:51.745138 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:51.745160 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:51.746120 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:51.746127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:51.746325 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:51.747162 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:51.747178 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:51.748324 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:51.752245 (+ 3921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:51.752249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:51.752251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:51.752868 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:51.753338 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:51.753346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:51.753353 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:51.753572 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:51.754224 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:51.754228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:51.754416 (+ 188us) client_negotiation.cc:769] Sending connection context
0621 06:57:51.754612 (+ 196us) client_negotiation.cc:240] Negotiation successful
0621 06:57:51.754996 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":360,"threads_started":1}
I20260621 06:57:51.763722 4837 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36991
I20260621 06:57:51.763746 4972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36991 every 8 connection(s)
I20260621 06:57:51.764885 4837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:57:51.766899 4973 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38275
I20260621 06:57:51.767160 4973 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:51.767771 4973 heartbeater.cc:511] Master 127.2.245.254:38275 requested a full tablet report, sending...
I20260621 06:57:51.769658 4766 ts_manager.cc:194] Registered new tserver with Master: d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991)
I20260621 06:57:51.771183 4766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:40003
I20260621 06:57:51.773777 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4837
I20260621 06:57:51.773972 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:57:51.774626 3031 external_mini_cluster.cc:1428] Setting key 70ef15a1f5d8f0fbaf66fdd0e71c85de
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:57:51.814124 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38275
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:52.131644 4977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:52.131956 4977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:52.132138 4977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:52.139394 4977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:52.139544 4977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:52.139703 4977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 06:57:52.139796 4977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:52.139931 4977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:57:52.149909 4977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:52.151382 4977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:52.152657 4977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:52.165258 4982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:52.165257 4983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:52.165943 4985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:52.166749 4977 server_base.cc:1034] running on GCE node
I20260621 06:57:52.167448 4977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:52.168470 4977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:52.169716 4977 hybrid_clock.cc:648] HybridClock initialized: now 1782025072169666 us; error 52 us; skew 500 ppm
I20260621 06:57:52.170030 4977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:52.173372 4977 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:57:52.174932 4977 webserver.cc:460] Webserver started at http://127.2.245.194:35231/ using document root <none> and password file <none>
I20260621 06:57:52.175657 4977 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:52.175804 4977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:52.176134 4977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:52.179361 4977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "2d838b90d7db454dab63db39497676b9"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "f24f9add2621b10f754d8539d6366d41"
server_key_iv: "b85a897e60b19c6d2e45aff29142686c"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.180160 4977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "2d838b90d7db454dab63db39497676b9"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "f24f9add2621b10f754d8539d6366d41"
server_key_iv: "b85a897e60b19c6d2e45aff29142686c"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.185531 4977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:57:52.189030 4992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:52.190426 4977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:57:52.190606 4977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "2d838b90d7db454dab63db39497676b9"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "f24f9add2621b10f754d8539d6366d41"
server_key_iv: "b85a897e60b19c6d2e45aff29142686c"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.190819 4977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:52.214638 4977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:52.218770 4977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:52.219123 4977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:52.221951 4977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:52.222077 4977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:52.222193 4977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:52.222287 4977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:52.247364 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.222710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:56047 (local address 127.2.245.254:38275)
0621 06:57:52.222911 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.222917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.224498 (+ 1581us) server_negotiation.cc:407] Connection header received
0621 06:57:52.226823 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.226829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.226923 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.227052 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:52.229438 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.230566 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.233352 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.233870 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.234044 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.239576 (+ 5532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:52.239581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:52.239584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:52.239618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:52.243409 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.244442 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.244451 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.244457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.244656 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.245126 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.245133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.245139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.245619 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:52.245838 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.246564 (+ 726us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.246863 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 06:57:52.248474 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.223447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.194:56047)
0621 06:57:52.224317 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.224365 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.226495 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.227330 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.227339 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.228119 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:52.229207 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.229246 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.230818 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.230825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.233203 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.233214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.234191 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.234203 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.234608 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.235568 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:52.235608 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:52.239295 (+ 3687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:52.243589 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.243604 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.243625 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.244270 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.244795 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.244798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.244800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.244988 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.245782 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:52.245788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:52.246139 (+ 351us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.246390 (+ 251us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.246787 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":341,"threads_started":1}
I20260621 06:57:52.271445 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.255628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:56837 (local address 127.2.245.254:38275)
0621 06:57:52.255880 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.255890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.256726 (+ 836us) server_negotiation.cc:407] Connection header received
0621 06:57:52.256920 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.256923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.256983 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.257099 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:52.258613 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.259557 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.261232 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.261999 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.262125 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.264609 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:52.264619 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:52.264624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:52.264697 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:52.268124 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.269138 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.269147 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.269152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.269331 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.269772 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.269781 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.269784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.270252 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:52.270410 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.270818 (+ 408us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.271085 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:57:52.271723 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.255692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.194:56837)
0621 06:57:52.256594 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.256617 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.256799 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.257131 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.257135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.257910 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:52.258455 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.258469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.259719 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.259729 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.261007 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.261023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.262200 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.262207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.262365 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.263044 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:52.263069 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:52.264215 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:52.268270 (+ 4055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.268279 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.268283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.269005 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.269474 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.269480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.269491 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.269628 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.270379 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:52.270384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:52.270569 (+ 185us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.270718 (+ 149us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.271103 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":545,"threads_started":1}
I20260621 06:57:52.279695 4977 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:36429
I20260621 06:57:52.279732 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:36429 every 8 connection(s)
I20260621 06:57:52.280947 4977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:57:52.282420 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 4977
I20260621 06:57:52.282620 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:57:52.283186 3031 external_mini_cluster.cc:1428] Setting key d865b0f70c0b9b255f67af13fc1c476b
I20260621 06:57:52.283246 5112 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38275
I20260621 06:57:52.283466 5112 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:52.284284 5112 heartbeater.cc:511] Master 127.2.245.254:38275 requested a full tablet report, sending...
I20260621 06:57:52.285645 4766 ts_manager.cc:194] Registered new tserver with Master: 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:52.286432 4766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:56047
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:57:52.324445 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38275
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:52.644662 5116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:52.644965 5116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:52.645135 5116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:52.652441 5116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:52.652588 5116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:52.652743 5116 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 06:57:52.652829 5116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:57:52.652935 5116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:57:52.662465 5116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:52.663908 5116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:52.665349 5116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:52.678486 5122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:52.678459 5121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:52.678540 5124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:52.679279 5116 server_base.cc:1034] running on GCE node
I20260621 06:57:52.679844 5116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:52.680709 5116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:52.681929 5116 hybrid_clock.cc:648] HybridClock initialized: now 1782025072681868 us; error 52 us; skew 500 ppm
I20260621 06:57:52.682278 5116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:52.685600 5116 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:57:52.687618 5116 webserver.cc:460] Webserver started at http://127.2.245.195:43865/ using document root <none> and password file <none>
I20260621 06:57:52.688315 5116 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:52.688457 5116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:52.688796 5116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:52.691783 5116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "40e49f77b7344cdcbeec0f2377a22797"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "d5d68d5d80eb4499d3a570133861586c"
server_key_iv: "6c67ccdc8ede928cb442e85347b55ce0"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.692725 5116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "40e49f77b7344cdcbeec0f2377a22797"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "d5d68d5d80eb4499d3a570133861586c"
server_key_iv: "6c67ccdc8ede928cb442e85347b55ce0"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.698825 5116 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260621 06:57:52.702260 5131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:52.703644 5116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:57:52.703823 5116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "40e49f77b7344cdcbeec0f2377a22797"
format_stamp: "Formatted at 2026-06-21 06:57:52 on dist-test-slave-9zdv"
server_key: "d5d68d5d80eb4499d3a570133861586c"
server_key_iv: "6c67ccdc8ede928cb442e85347b55ce0"
server_key_version: "encryptionkey@0"
I20260621 06:57:52.704032 5116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:57:52.724947 5116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:57:52.729143 5116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:57:52.729457 5116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:57:52.732168 5116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:57:52.732316 5116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:57:52.732445 5116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:57:52.732549 5116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:52.756489 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.732576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50109 (local address 127.2.245.254:38275)
0621 06:57:52.732823 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.732833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.734358 (+ 1525us) server_negotiation.cc:407] Connection header received
0621 06:57:52.736621 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.736627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.736726 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.736852 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:52.739003 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.740331 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.742997 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.743633 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.743782 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.748922 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:52.748931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:52.748936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:52.748982 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:52.752662 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.753697 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.753702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.753706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.753879 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.754447 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.754454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.754460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.754748 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:52.754909 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.755893 (+ 984us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.756136 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 06:57:52.757664 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.733238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.195:50109)
0621 06:57:52.734168 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.734218 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.736288 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.737139 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.737157 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.737938 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:52.738763 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.738797 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.740514 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.740523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.742830 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.742843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.743877 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.743884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.744086 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.744889 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:52.744935 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:52.748580 (+ 3645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:52.752837 (+ 4257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.752848 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.752873 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.753539 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.754040 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.754047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.754052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.754319 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.754886 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:52.754894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:52.755351 (+ 457us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.755800 (+ 449us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.756392 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":434,"threads_started":1}
I20260621 06:57:52.773906 4973 heartbeater.cc:503] Master 127.2.245.254:38275 was elected leader, sending a full tablet report...
I20260621 06:57:52.787266 5116 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:46845
I20260621 06:57:52.787323 5250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:46845 every 8 connection(s)
I20260621 06:57:52.788506 5116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:57:52.790618 5251 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38275
I20260621 06:57:52.790839 5251 heartbeater.cc:464] Registering TS with master...
I20260621 06:57:52.791512 5251 heartbeater.cc:511] Master 127.2.245.254:38275 requested a full tablet report, sending...
I20260621 06:57:52.793159 4766 ts_manager.cc:194] Registered new tserver with Master: 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845)
I20260621 06:57:52.793262 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5116
I20260621 06:57:52.793443 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:57:52.793992 3031 external_mini_cluster.cc:1428] Setting key fffca777aac16eb3f98f5a39124b7246
I20260621 06:57:52.794426 4766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:50109
I20260621 06:57:52.797430 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:57:52.799041 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.774708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:56843 (local address 127.2.245.254:38275)
0621 06:57:52.774904 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.774910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.779743 (+ 4833us) server_negotiation.cc:407] Connection header received
0621 06:57:52.779939 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.779944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.780022 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.780137 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:52.781867 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.783176 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.785769 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.786397 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.786518 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.793086 (+ 6568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:52.793097 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:52.793100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:52.793133 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:52.796447 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.797235 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.797240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.797243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.797399 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.797792 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.797795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.797797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.798033 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:52.798151 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.798567 (+ 416us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.798738 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260621 06:57:52.799219 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.775272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.2.245.195:56843)
0621 06:57:52.779599 (+ 4327us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.779624 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.779795 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.780166 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.780171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.781034 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:52.781656 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.781674 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.783371 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.783383 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.785604 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.785620 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.786595 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.786602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.786767 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.791915 (+ 5148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:52.791936 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:52.792918 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:52.796592 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.796598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.796601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.797105 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.797535 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:52.797540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:52.797543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:52.797679 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:52.798148 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:52.798154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:52.798346 (+ 192us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.798488 (+ 142us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.798777 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4104,"thread_start_us":3906,"threads_started":1}
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 06:57:52 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:52.843917 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.830882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.2.245.254:38275)
0621 06:57:52.831065 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.831069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.831440 (+ 371us) server_negotiation.cc:407] Connection header received
0621 06:57:52.831590 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.831594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.831659 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.831762 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:52.833070 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.833801 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.834993 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.835450 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.835542 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.839193 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:52.839198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:52.839201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:52.839237 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:52.841672 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.842254 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.842257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.842260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.842373 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:52.842702 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:52.842705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:52.842707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:52.843044 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:52.843231 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.843471 (+ 240us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.843650 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260621 06:57:52.847878 4766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:52.854696 4766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:52.892684 5265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.866524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46845 (local address 127.0.0.1:33222)
0621 06:57:52.879766 (+ 13242us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.879821 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.880065 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.885756 (+ 5691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.885760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.885790 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:52.886331 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.886342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887847 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.887856 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.889659 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.889671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.891585 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.891592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.891798 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.891850 (+ 52us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.891994 (+ 144us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.892206 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12962,"mutex_wait_us":106,"thread_start_us":16708,"threads_started":1}
I20260621 06:57:52.893088 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.869833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.2.245.194:36429)
0621 06:57:52.883878 (+ 14045us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.883887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.884587 (+ 700us) server_negotiation.cc:407] Connection header received
0621 06:57:52.884703 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.884711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.885038 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.885248 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:52.886052 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887536 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.890464 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.891834 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.891985 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.892189 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.892431 (+ 242us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.892603 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13884,"thread_start_us":13750,"threads_started":1}
I20260621 06:57:52.893349 5268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.869202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36429 (local address 127.0.0.1:52410)
0621 06:57:52.884332 (+ 15130us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.884350 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.884464 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.885278 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.885286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.885325 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:52.885843 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.885858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887684 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.887690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.890297 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.890313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.892043 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.892051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.892205 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.892250 (+ 45us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.892380 (+ 130us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.892508 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15004,"mutex_wait_us":119,"thread_start_us":14856,"threads_started":1}
I20260621 06:57:52.893560 5266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.866824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.2.245.195:46845)
0621 06:57:52.884958 (+ 18134us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.884968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.884999 (+ 31us) server_negotiation.cc:407] Connection header received
0621 06:57:52.885080 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.885085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.885350 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.885553 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:52.886545 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887710 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.889831 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.891384 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.891519 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.891755 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.892782 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.893052 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17957,"thread_start_us":17734,"threads_started":1}
I20260621 06:57:52.893971 5267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.867267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.2.245.193:36991)
0621 06:57:52.882784 (+ 15517us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:52.882797 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:52.883677 (+ 880us) server_negotiation.cc:407] Connection header received
0621 06:57:52.885031 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:52.885039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:52.885260 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:52.885455 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:52.886376 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887858 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.890999 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.892277 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.892419 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.892618 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:52.893456 (+ 838us) server_negotiation.cc:299] Negotiation successful
0621 06:57:52.893656 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15277,"thread_start_us":15523,"threads_started":1}
I20260621 06:57:52.894416 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:52.866655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36991 (local address 127.0.0.1:37288)
0621 06:57:52.883543 (+ 16888us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:52.883567 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:52.883793 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:52.885506 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:52.885515 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:52.885540 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:52.886181 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.886198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.887993 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.888000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.890830 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:52.890844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:52.892791 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:52.892799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:52.892997 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:52.893043 (+ 46us) client_negotiation.cc:769] Sending connection context
0621 06:57:52.893128 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 06:57:52.893263 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16721,"mutex_wait_us":114,"thread_start_us":12658,"threads_started":1}
I20260621 06:57:52.897409 5181 tablet_service.cc:1467] Processing CreateTablet for tablet 91cacb4168d746749612c23fd5008777 (DEFAULT_TABLE table=test-table [id=75fc1e68fd374d479c901c4690bcac65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:52.897413 5042 tablet_service.cc:1467] Processing CreateTablet for tablet 91cacb4168d746749612c23fd5008777 (DEFAULT_TABLE table=test-table [id=75fc1e68fd374d479c901c4690bcac65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:52.898978 5181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91cacb4168d746749612c23fd5008777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:52.899595 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91cacb4168d746749612c23fd5008777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:52.900693 4903 tablet_service.cc:1467] Processing CreateTablet for tablet 91cacb4168d746749612c23fd5008777 (DEFAULT_TABLE table=test-table [id=75fc1e68fd374d479c901c4690bcac65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:52.902016 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91cacb4168d746749612c23fd5008777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:52.913539 5271 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Bootstrap starting.
I20260621 06:57:52.915370 5270 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Bootstrap starting.
I20260621 06:57:52.917723 5272 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Bootstrap starting.
I20260621 06:57:52.918233 5271 tablet_bootstrap.cc:654] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:52.918656 5270 tablet_bootstrap.cc:654] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:52.919528 5271 log.cc:824] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:52.919672 5270 log.cc:824] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:52.920955 5272 tablet_bootstrap.cc:654] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:52.922176 5272 log.cc:824] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Log is configured to *not* fsync() on all Append() calls
I20260621 06:57:52.923096 5271 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: No bootstrap required, opened a new log
I20260621 06:57:52.923339 5270 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: No bootstrap required, opened a new log
I20260621 06:57:52.923369 5271 ts_tablet_manager.cc:1397] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260621 06:57:52.923597 5270 ts_tablet_manager.cc:1397] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260621 06:57:52.925076 5272 tablet_bootstrap.cc:492] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: No bootstrap required, opened a new log
I20260621 06:57:52.925313 5272 ts_tablet_manager.cc:1397] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260621 06:57:52.931865 5271 raft_consensus.cc:348] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.932260 5271 raft_consensus.cc:374] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:52.932359 5271 raft_consensus.cc:729] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Initialized, Role: FOLLOWER
I20260621 06:57:52.932988 5271 consensus_queue.cc:260] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.934055 5251 heartbeater.cc:503] Master 127.2.245.254:38275 was elected leader, sending a full tablet report...
I20260621 06:57:52.934262 5270 raft_consensus.cc:348] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.934675 5270 raft_consensus.cc:374] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:52.934770 5270 raft_consensus.cc:729] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Initialized, Role: FOLLOWER
I20260621 06:57:52.935372 5270 consensus_queue.cc:260] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.936177 5271 ts_tablet_manager.cc:1428] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260621 06:57:52.935842 5272 raft_consensus.cc:348] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.936322 5272 raft_consensus.cc:374] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:52.936430 5272 raft_consensus.cc:729] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d379eacb6d6e45acb1d331b935a9f9be, State: Initialized, Role: FOLLOWER
I20260621 06:57:52.937286 5112 heartbeater.cc:503] Master 127.2.245.254:38275 was elected leader, sending a full tablet report...
I20260621 06:57:52.937150 5272 consensus_queue.cc:260] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:52.937940 5270 ts_tablet_manager.cc:1428] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260621 06:57:52.939467 5272 ts_tablet_manager.cc:1428] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260621 06:57:53.017493 4974 tablet.cc:2369] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:57:53.033082 5113 tablet.cc:2369] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:57:53.040127 5252 tablet.cc:2369] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:57:53.207872 5276 raft_consensus.cc:482] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:53.208161 5276 raft_consensus.cc:504] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:53.209966 5276 leader_election.cc:288] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.217142 5267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.210297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:51037 (local address 127.2.245.193:36991)
0621 06:57:53.210484 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.210491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.210602 (+ 111us) server_negotiation.cc:407] Connection header received
0621 06:57:53.210809 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.210813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.210889 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.211027 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:53.211737 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.212713 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.215176 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.216341 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.216513 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.216593 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.216794 (+ 201us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.216930 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 06:57:53.217224 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.210190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36991 (local address 127.2.245.195:51037)
0621 06:57:53.210454 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:53.210474 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:53.210625 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:53.211046 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:53.211051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:53.211076 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:53.211331 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.211343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.212841 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.212845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.215012 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.215026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.216487 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.216491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.216617 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.216641 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 06:57:53.216763 (+ 122us) client_negotiation.cc:240] Negotiation successful
0621 06:57:53.216911 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":42}
I20260621 06:57:53.217554 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.210426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50651 (local address 127.2.245.194:36429)
0621 06:57:53.210625 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.210633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.211341 (+ 708us) server_negotiation.cc:407] Connection header received
0621 06:57:53.211468 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.211473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.211540 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.211609 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:53.212155 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.213091 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.215451 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.216628 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.216806 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.216870 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.217170 (+ 300us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.217319 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 06:57:53.217607 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.210478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36429 (local address 127.2.245.195:50651)
0621 06:57:53.211236 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:53.211254 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:53.211362 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:53.211636 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:53.211640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:53.211657 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:53.211965 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.211972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.213225 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.213230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.215312 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.215321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.216840 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.216846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.216992 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.217030 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 06:57:53.217117 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 06:57:53.217236 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":42,"thread_start_us":380,"threads_started":1}
I20260621 06:57:53.218595 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91cacb4168d746749612c23fd5008777" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be" is_pre_election: true
I20260621 06:57:53.218593 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91cacb4168d746749612c23fd5008777" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9" is_pre_election: true
I20260621 06:57:53.219121 4928 raft_consensus.cc:2454] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.219121 5065 raft_consensus.cc:2454] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.219805 5135 leader_election.cc:302] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:53.220213 5276 raft_consensus.cc:2781] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:53.220377 5276 raft_consensus.cc:482] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:53.220496 5276 raft_consensus.cc:3037] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.222657 5276 raft_consensus.cc:504] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:53.223333 5276 leader_election.cc:288] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Requested vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.223758 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91cacb4168d746749612c23fd5008777" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be"
I20260621 06:57:53.223860 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91cacb4168d746749612c23fd5008777" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9"
I20260621 06:57:53.223984 4928 raft_consensus.cc:3037] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.224030 5065 raft_consensus.cc:3037] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.227330 5065 raft_consensus.cc:2454] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.227330 4928 raft_consensus.cc:2454] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.227749 5135 leader_election.cc:302] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:53.228106 5276 raft_consensus.cc:2781] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:53.228503 5276 raft_consensus.cc:686] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Becoming Leader. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Running, Role: LEADER
I20260621 06:57:53.229003 5276 consensus_queue.cc:237] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } }
I20260621 06:57:53.233497 4765 catalog_manager.cc:5495] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } health_report { overall_health: UNKNOWN } } }
Jun 21 06:57:53 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025071, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:53.354429 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.341212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.2.245.254:38275)
0621 06:57:53.341387 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.341391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.341951 (+ 560us) server_negotiation.cc:407] Connection header received
0621 06:57:53.342155 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.342160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.342245 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.342364 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:53.343550 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.344254 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.345498 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.345914 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.346053 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.349257 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:53.349262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:53.349265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:53.349293 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:53.351802 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:53.352471 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:53.352477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:53.352484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:53.352611 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:53.352965 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:53.352969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:53.352973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:53.353239 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:53.353398 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.353793 (+ 395us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.354001 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 06:57:53.354557 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.341270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38275 (local address 127.0.0.1:46190)
0621 06:57:53.341829 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:53.341858 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:53.342032 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:53.342383 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:53.342388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:53.343154 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:57:53.343405 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.343416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.344410 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.344416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.345359 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.345370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.346098 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.346104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.346257 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.346876 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:57:53.346898 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:57:53.349069 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:57:53.351926 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:53.351941 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:53.351948 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:53.352354 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:53.352716 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:57:53.352722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:57:53.352725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:57:53.352855 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:57:53.353380 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:57:53.353387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:57:53.353553 (+ 166us) client_negotiation.cc:769] Sending connection context
0621 06:57:53.353713 (+ 160us) client_negotiation.cc:240] Negotiation successful
0621 06:57:53.354126 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":141,"thread_start_us":302,"threads_started":1}
I20260621 06:57:53.359539 4765 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:46190:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 06:57:53.360595 4765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:53.367430 4903 tablet_service.cc:1467] Processing CreateTablet for tablet 711b6653676e44ebbaf7d205a3ab0f82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7ae1f26d9847e1a550e4833cf78b94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:53.367911 5042 tablet_service.cc:1467] Processing CreateTablet for tablet 711b6653676e44ebbaf7d205a3ab0f82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7ae1f26d9847e1a550e4833cf78b94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:53.368481 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711b6653676e44ebbaf7d205a3ab0f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.368075 5181 tablet_service.cc:1467] Processing CreateTablet for tablet 711b6653676e44ebbaf7d205a3ab0f82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7ae1f26d9847e1a550e4833cf78b94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:57:53.368746 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711b6653676e44ebbaf7d205a3ab0f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.368906 5181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711b6653676e44ebbaf7d205a3ab0f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.374576 5270 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: Bootstrap starting.
I20260621 06:57:53.375810 5271 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: Bootstrap starting.
I20260621 06:57:53.376082 5270 tablet_bootstrap.cc:654] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.376423 5272 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: Bootstrap starting.
I20260621 06:57:53.377701 5271 tablet_bootstrap.cc:654] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.377772 5272 tablet_bootstrap.cc:654] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.379853 5272 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: No bootstrap required, opened a new log
I20260621 06:57:53.380017 5272 ts_tablet_manager.cc:1397] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260621 06:57:53.380174 5270 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: No bootstrap required, opened a new log
I20260621 06:57:53.380174 5271 tablet_bootstrap.cc:492] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: No bootstrap required, opened a new log
I20260621 06:57:53.380319 5271 ts_tablet_manager.cc:1397] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260621 06:57:53.380319 5270 ts_tablet_manager.cc:1397] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 06:57:53.380824 5272 raft_consensus.cc:348] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.381075 5272 raft_consensus.cc:374] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.381171 5272 raft_consensus.cc:729] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d379eacb6d6e45acb1d331b935a9f9be, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.381042 5271 raft_consensus.cc:348] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.381039 5270 raft_consensus.cc:348] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.381273 5270 raft_consensus.cc:374] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.381268 5271 raft_consensus.cc:374] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.381374 5271 raft_consensus.cc:729] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.381386 5270 raft_consensus.cc:729] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.381397 5272 consensus_queue.cc:260] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.381601 5270 consensus_queue.cc:260] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.381644 5271 consensus_queue.cc:260] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.382017 5272 ts_tablet_manager.cc:1428] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260621 06:57:53.382273 5270 ts_tablet_manager.cc:1428] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260621 06:57:53.382107 5277 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.382498 5277 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.382316 5278 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.382664 5278 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.382830 5276 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.383167 5276 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.383671 5271 ts_tablet_manager.cc:1428] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260621 06:57:53.596401 5276 raft_consensus.cc:482] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:53.596649 5276 raft_consensus.cc:504] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.597295 5276 leader_election.cc:288] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.597971 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711b6653676e44ebbaf7d205a3ab0f82" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be" is_pre_election: true
I20260621 06:57:53.597971 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711b6653676e44ebbaf7d205a3ab0f82" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9" is_pre_election: true
I20260621 06:57:53.598273 5065 raft_consensus.cc:2454] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.598284 4928 raft_consensus.cc:2454] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.598737 5134 leader_election.cc:302] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:53.599184 5276 raft_consensus.cc:2781] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:53.599275 5276 raft_consensus.cc:482] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:53.599426 5276 raft_consensus.cc:3037] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.601413 5276 raft_consensus.cc:504] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.601954 5276 leader_election.cc:288] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Requested vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.602550 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711b6653676e44ebbaf7d205a3ab0f82" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be"
I20260621 06:57:53.602550 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711b6653676e44ebbaf7d205a3ab0f82" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9"
I20260621 06:57:53.602803 5065 raft_consensus.cc:3037] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.602803 4928 raft_consensus.cc:3037] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.605791 4928 raft_consensus.cc:2454] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.605791 5065 raft_consensus.cc:2454] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.606230 5134 leader_election.cc:302] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:53.606552 5276 raft_consensus.cc:2781] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:53.606734 5276 raft_consensus.cc:686] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Becoming Leader. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Running, Role: LEADER
I20260621 06:57:53.607025 5276 consensus_queue.cc:237] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.608536 5280 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: TxnStatusTablet state changed. Reason: New leader 40e49f77b7344cdcbeec0f2377a22797. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.608860 5280 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:53.609469 5292 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 06:57:53.609779 5292 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 06:57:53.610105 4764 catalog_manager.cc:5495] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } health_report { overall_health: HEALTHY } } }
I20260621 06:57:53.675623 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.669383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46845 (local address 127.0.0.1:33230)
0621 06:57:53.669600 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:53.669624 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:53.669785 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:53.670414 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:53.670418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:53.670430 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:53.670649 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.670659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.671874 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.671882 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.672882 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.672894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.673677 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.673684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.673846 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.673901 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:53.674993 (+ 1092us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:53.675031 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 06:57:53.675141 (+ 110us) client_negotiation.cc:240] Negotiation successful
0621 06:57:53.675297 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260621 06:57:53.675693 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.669465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.2.245.195:46845)
0621 06:57:53.670038 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.670043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.670061 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:57:53.670138 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.670141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.670220 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.670395 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:53.670842 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.671724 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.673043 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.673509 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.673660 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.674132 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:53.674841 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:53.675044 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.675229 (+ 185us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.675305 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":267,"threads_started":1}
I20260621 06:57:53.683997 4928 raft_consensus.cc:1261] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:53.684020 5065 raft_consensus.cc:1261] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:53.684950 5276 consensus_queue.cc:1035] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:53.684918 5280 consensus_queue.cc:1035] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:53.685679 5297 consensus_queue.cc:1035] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:53.706704 5278 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: TxnStatusTablet state changed. Reason: New leader 40e49f77b7344cdcbeec0f2377a22797. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.707046 5278 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.707768 5277 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: TxnStatusTablet state changed. Reason: New leader 40e49f77b7344cdcbeec0f2377a22797. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.708048 5277 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.711632 5280 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.711928 5280 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:53.712157 5280 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.712410 5280 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 40e49f77b7344cdcbeec0f2377a22797: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:57:53.713795 5299 mvcc.cc:205] Tried to move back new op lower bound from 7299174701799280640 to 7299174701496180736. Current Snapshot: MvccSnapshot[applied={T|T < 7299174701799280640}]
I20260621 06:57:53.714010 5298 mvcc.cc:205] Tried to move back new op lower bound from 7299174701799280640 to 7299174701496180736. Current Snapshot: MvccSnapshot[applied={T|T < 7299174701799280640}]
I20260621 06:57:53.714728 5296 mvcc.cc:205] Tried to move back new op lower bound from 7299174701799280640 to 7299174701496180736. Current Snapshot: MvccSnapshot[applied={T|T < 7299174701799280640}]
I20260621 06:57:53.716812 5278 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.717133 5278 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P d379eacb6d6e45acb1d331b935a9f9be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.718539 5277 tablet_replica.cc:439] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } } }
I20260621 06:57:53.718766 5277 tablet_replica.cc:442] T 711b6653676e44ebbaf7d205a3ab0f82 P 2d838b90d7db454dab63db39497676b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:57:53.724947 5276 consensus_queue.cc:1035] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:53.737169 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.730129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.2.245.195:46845)
0621 06:57:53.730346 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.730355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.731418 (+ 1063us) server_negotiation.cc:407] Connection header received
0621 06:57:53.731703 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.731708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.731789 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.731895 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:53.732402 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.733583 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.735203 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.735872 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.735990 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.736421 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:53.736554 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:53.736653 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.736774 (+ 121us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.736865 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 06:57:53.754616 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.748059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46845 (local address 127.2.245.195:50165)
0621 06:57:53.748898 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:53.748913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:53.749027 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:53.749367 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:53.749370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:53.749378 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:53.749671 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.749678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.751126 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.751129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.752486 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:53.752503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.753459 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.753464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:53.753642 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.753694 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:53.754098 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:53.754106 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 06:57:53.754164 (+ 58us) client_negotiation.cc:240] Negotiation successful
0621 06:57:53.754259 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":2,"thread_start_us":479,"threads_started":1}
I20260621 06:57:53.754640 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:53.748082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50165 (local address 127.2.245.195:46845)
0621 06:57:53.748240 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:53.748247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:53.749003 (+ 756us) server_negotiation.cc:407] Connection header received
0621 06:57:53.749159 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:53.749164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:53.749256 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:53.749359 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:53.749807 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.750975 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.752639 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:53.753312 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:53.753446 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:53.753879 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:53.754000 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:53.754092 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:53.754215 (+ 123us) server_negotiation.cc:299] Negotiation successful
0621 06:57:53.754297 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":52}
I20260621 06:57:53.847393 4765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:53.847677 4765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:53.851500 4765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Sending DeleteTablet for 3 replicas of tablet 91cacb4168d746749612c23fd5008777
I20260621 06:57:53.852721 4903 tablet_service.cc:1514] Processing DeleteTablet for tablet 91cacb4168d746749612c23fd5008777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:53 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37288
I20260621 06:57:53.852967 5042 tablet_service.cc:1514] Processing DeleteTablet for tablet 91cacb4168d746749612c23fd5008777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:53 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52410
I20260621 06:57:53.854105 4765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:53.855196 4765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:53.856300 5181 tablet_service.cc:1514] Processing DeleteTablet for tablet 91cacb4168d746749612c23fd5008777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:53 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33222
I20260621 06:57:53.863040 5180 tablet_service.cc:1467] Processing CreateTablet for tablet 615a9953a2234e4e973da1893b0abed0 (DEFAULT_TABLE table=test-table [id=7e80762238b14a829b857011ec49de34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:53.863536 5315 tablet_replica.cc:330] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: stopping tablet replica
I20260621 06:57:53.863571 5180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615a9953a2234e4e973da1893b0abed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.863688 5314 tablet_replica.cc:330] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: stopping tablet replica
I20260621 06:57:53.864032 5314 raft_consensus.cc:2229] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:53.864032 5315 raft_consensus.cc:2229] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:53.864406 5315 raft_consensus.cc:2258] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:53.864416 5314 raft_consensus.cc:2258] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:53.865846 5042 tablet_service.cc:1467] Processing CreateTablet for tablet 615a9953a2234e4e973da1893b0abed0 (DEFAULT_TABLE table=test-table [id=7e80762238b14a829b857011ec49de34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:53.866258 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615a9953a2234e4e973da1893b0abed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.866384 5314 ts_tablet_manager.cc:1905] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:53.870133 5271 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Bootstrap starting.
I20260621 06:57:53.871865 5271 tablet_bootstrap.cc:654] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.872663 5315 ts_tablet_manager.cc:1905] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:53.873054 5270 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Bootstrap starting.
I20260621 06:57:53.874277 4903 tablet_service.cc:1467] Processing CreateTablet for tablet 615a9953a2234e4e973da1893b0abed0 (DEFAULT_TABLE table=test-table [id=7e80762238b14a829b857011ec49de34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:53.874704 5270 tablet_bootstrap.cc:654] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.875656 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615a9953a2234e4e973da1893b0abed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:53.881932 5314 ts_tablet_manager.cc:1918] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:53.882164 5314 log.cc:1196] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/91cacb4168d746749612c23fd5008777
I20260621 06:57:53.882809 5314 ts_tablet_manager.cc:1939] T 91cacb4168d746749612c23fd5008777 P d379eacb6d6e45acb1d331b935a9f9be: Deleting consensus metadata
I20260621 06:57:53.885293 4752 catalog_manager.cc:4841] TS d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991): tablet 91cacb4168d746749612c23fd5008777 (table test-table [id=75fc1e68fd374d479c901c4690bcac65]) successfully deleted
I20260621 06:57:53.887686 5315 ts_tablet_manager.cc:1918] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:53.887879 5315 log.cc:1196] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/91cacb4168d746749612c23fd5008777
I20260621 06:57:53.888430 5315 ts_tablet_manager.cc:1939] T 91cacb4168d746749612c23fd5008777 P 2d838b90d7db454dab63db39497676b9: Deleting consensus metadata
I20260621 06:57:53.888484 5316 tablet_replica.cc:330] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: stopping tablet replica
I20260621 06:57:53.889081 5316 raft_consensus.cc:2229] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:53.890012 5316 raft_consensus.cc:2258] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:53.890153 4753 catalog_manager.cc:4841] TS 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429): tablet 91cacb4168d746749612c23fd5008777 (table test-table [id=75fc1e68fd374d479c901c4690bcac65]) successfully deleted
I20260621 06:57:53.892689 5316 ts_tablet_manager.cc:1905] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:53.893868 5319 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Bootstrap starting.
I20260621 06:57:53.896142 5319 tablet_bootstrap.cc:654] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:53.902145 5316 ts_tablet_manager.cc:1918] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:53.902245 5270 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: No bootstrap required, opened a new log
I20260621 06:57:53.902333 5316 log.cc:1196] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/91cacb4168d746749612c23fd5008777
I20260621 06:57:53.902426 5270 ts_tablet_manager.cc:1397] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.007s
I20260621 06:57:53.903090 5316 ts_tablet_manager.cc:1939] T 91cacb4168d746749612c23fd5008777 P 40e49f77b7344cdcbeec0f2377a22797: Deleting consensus metadata
I20260621 06:57:53.903290 5270 raft_consensus.cc:348] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.903561 5270 raft_consensus.cc:374] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.903656 5270 raft_consensus.cc:729] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.904991 4750 catalog_manager.cc:4841] TS 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845): tablet 91cacb4168d746749612c23fd5008777 (table test-table [id=75fc1e68fd374d479c901c4690bcac65]) successfully deleted
I20260621 06:57:53.905274 5319 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: No bootstrap required, opened a new log
I20260621 06:57:53.905489 5319 ts_tablet_manager.cc:1397] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260621 06:57:53.905385 5270 consensus_queue.cc:260] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.906172 5270 ts_tablet_manager.cc:1428] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260621 06:57:53.906267 5319 raft_consensus.cc:348] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.906570 5319 raft_consensus.cc:374] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.906665 5319 raft_consensus.cc:729] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d379eacb6d6e45acb1d331b935a9f9be, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.906948 5319 consensus_queue.cc:260] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.907402 5271 tablet_bootstrap.cc:492] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: No bootstrap required, opened a new log
I20260621 06:57:53.907604 5271 ts_tablet_manager.cc:1397] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.009s
I20260621 06:57:53.908409 5271 raft_consensus.cc:348] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.908654 5271 raft_consensus.cc:374] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:53.908746 5271 raft_consensus.cc:729] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Initialized, Role: FOLLOWER
I20260621 06:57:53.908968 5271 consensus_queue.cc:260] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.910972 5319 ts_tablet_manager.cc:1428] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260621 06:57:53.911573 5271 ts_tablet_manager.cc:1428] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260621 06:57:53.945173 5276 raft_consensus.cc:482] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:53.945402 5276 raft_consensus.cc:504] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.946204 5276 leader_election.cc:288] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.946760 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615a9953a2234e4e973da1893b0abed0" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9" is_pre_election: true
I20260621 06:57:53.946820 4927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615a9953a2234e4e973da1893b0abed0" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be" is_pre_election: true
I20260621 06:57:53.947039 5065 raft_consensus.cc:2454] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.947121 4927 raft_consensus.cc:2454] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:53.947547 5135 leader_election.cc:302] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797; no voters:
I20260621 06:57:53.947841 5276 raft_consensus.cc:2781] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:53.947993 5276 raft_consensus.cc:482] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:53.948117 5276 raft_consensus.cc:3037] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.950335 5276 raft_consensus.cc:504] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.951455 5276 leader_election.cc:288] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Requested vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:53.951759 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615a9953a2234e4e973da1893b0abed0" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9"
I20260621 06:57:53.951782 4927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615a9953a2234e4e973da1893b0abed0" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be"
I20260621 06:57:53.952023 5065 raft_consensus.cc:3037] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.952013 4927 raft_consensus.cc:3037] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:53.953917 5065 raft_consensus.cc:2454] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.954382 5135 leader_election.cc:302] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797; no voters:
I20260621 06:57:53.954679 5276 raft_consensus.cc:2781] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:53.954861 5276 raft_consensus.cc:686] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Becoming Leader. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Running, Role: LEADER
I20260621 06:57:53.955037 4927 raft_consensus.cc:2454] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:53.955183 5276 consensus_queue.cc:237] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:53.958154 4764 catalog_manager.cc:5495] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } health_report { overall_health: HEALTHY } } }
I20260621 06:57:53.991978 4928 raft_consensus.cc:1261] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:53.992065 5065 raft_consensus.cc:1261] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:53.992736 5313 consensus_queue.cc:1035] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:53.993079 5280 consensus_queue.cc:1035] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 06:57:54.018291 5296 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260621 06:57:55.601854 4765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:55.602090 4765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:55.605988 4765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Sending DeleteTablet for 3 replicas of tablet 615a9953a2234e4e973da1893b0abed0
I20260621 06:57:55.606730 4903 tablet_service.cc:1514] Processing DeleteTablet for tablet 615a9953a2234e4e973da1893b0abed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:55 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37288
I20260621 06:57:55.606843 5042 tablet_service.cc:1514] Processing DeleteTablet for tablet 615a9953a2234e4e973da1893b0abed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:55 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52410
I20260621 06:57:55.607002 5180 tablet_service.cc:1514] Processing DeleteTablet for tablet 615a9953a2234e4e973da1893b0abed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:55 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33222
I20260621 06:57:55.620240 5345 tablet_replica.cc:330] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: stopping tablet replica
I20260621 06:57:55.620492 5345 raft_consensus.cc:2229] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:55.620555 5343 tablet_replica.cc:330] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: stopping tablet replica
I20260621 06:57:55.620903 5345 raft_consensus.cc:2258] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:55.620909 5343 raft_consensus.cc:2229] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:55.621189 5343 raft_consensus.cc:2258] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:55.621062 5344 tablet_replica.cc:330] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: stopping tablet replica
I20260621 06:57:55.621537 5344 raft_consensus.cc:2229] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:55.621778 5344 raft_consensus.cc:2258] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:55.622080 5345 ts_tablet_manager.cc:1905] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:55.622138 5343 ts_tablet_manager.cc:1905] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:55.622767 5344 ts_tablet_manager.cc:1905] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:55.629138 5345 ts_tablet_manager.cc:1918] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:55.629129 5343 ts_tablet_manager.cc:1918] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:55.629303 5343 log.cc:1196] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/615a9953a2234e4e973da1893b0abed0
I20260621 06:57:55.629467 5345 log.cc:1196] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/615a9953a2234e4e973da1893b0abed0
I20260621 06:57:55.629806 5343 ts_tablet_manager.cc:1939] T 615a9953a2234e4e973da1893b0abed0 P d379eacb6d6e45acb1d331b935a9f9be: Deleting consensus metadata
I20260621 06:57:55.630002 5345 ts_tablet_manager.cc:1939] T 615a9953a2234e4e973da1893b0abed0 P 40e49f77b7344cdcbeec0f2377a22797: Deleting consensus metadata
I20260621 06:57:55.630568 5344 ts_tablet_manager.cc:1918] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:55.630724 5344 log.cc:1196] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/615a9953a2234e4e973da1893b0abed0
I20260621 06:57:55.631086 4752 catalog_manager.cc:4841] TS d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991): tablet 615a9953a2234e4e973da1893b0abed0 (table test-table [id=7e80762238b14a829b857011ec49de34]) successfully deleted
I20260621 06:57:55.631194 5344 ts_tablet_manager.cc:1939] T 615a9953a2234e4e973da1893b0abed0 P 2d838b90d7db454dab63db39497676b9: Deleting consensus metadata
I20260621 06:57:55.631258 4750 catalog_manager.cc:4841] TS 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845): tablet 615a9953a2234e4e973da1893b0abed0 (table test-table [id=7e80762238b14a829b857011ec49de34]) successfully deleted
I20260621 06:57:55.632174 4753 catalog_manager.cc:4841] TS 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429): tablet 615a9953a2234e4e973da1893b0abed0 (table test-table [id=7e80762238b14a829b857011ec49de34]) successfully deleted
Jun 21 06:57:55 dist-test-slave-9zdv krb5kdc[4721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 06:57:55 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:57:55.665477 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:55.651686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.2.245.254:38275)
0621 06:57:55.652277 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:55.652281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:55.652304 (+ 23us) server_negotiation.cc:407] Connection header received
0621 06:57:55.652453 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:55.652456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:55.652519 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:55.652638 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:55.653880 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:55.654685 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:55.655894 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:55.656284 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:55.656379 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:55.660523 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:55.660527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:55.660531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:55.660564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:55.663371 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:55.663950 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:55.663954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:55.663957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:55.664073 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:55.664398 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:55.664402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:55.664404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:55.664671 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:55.664846 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:55.665037 (+ 191us) server_negotiation.cc:299] Negotiation successful
0621 06:57:55.665213 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":351,"threads_started":1}
I20260621 06:57:55.668864 4765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:55.669487 4765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:55.675832 4903 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 (DEFAULT_TABLE table=test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:55.676079 5042 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 (DEFAULT_TABLE table=test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:55.676054 5180 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 (DEFAULT_TABLE table=test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:55.676318 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4253ad1b244a6cb9950bce708c3730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:55.676494 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4253ad1b244a6cb9950bce708c3730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:55.676585 5180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4253ad1b244a6cb9950bce708c3730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:55.682503 5354 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Bootstrap starting.
I20260621 06:57:55.683398 5355 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Bootstrap starting.
I20260621 06:57:55.683841 5356 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Bootstrap starting.
I20260621 06:57:55.684548 5354 tablet_bootstrap.cc:654] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:55.685187 5355 tablet_bootstrap.cc:654] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:55.685503 5356 tablet_bootstrap.cc:654] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:55.691028 5356 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: No bootstrap required, opened a new log
I20260621 06:57:55.691233 5356 ts_tablet_manager.cc:1397] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260621 06:57:55.691658 5354 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: No bootstrap required, opened a new log
I20260621 06:57:55.691816 5354 ts_tablet_manager.cc:1397] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260621 06:57:55.692190 5356 raft_consensus.cc:348] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.692430 5356 raft_consensus.cc:374] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:55.692533 5356 raft_consensus.cc:729] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d379eacb6d6e45acb1d331b935a9f9be, State: Initialized, Role: FOLLOWER
I20260621 06:57:55.692760 5356 consensus_queue.cc:260] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.692724 5354 raft_consensus.cc:348] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.692983 5354 raft_consensus.cc:374] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:55.693085 5354 raft_consensus.cc:729] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Initialized, Role: FOLLOWER
I20260621 06:57:55.693802 5355 tablet_bootstrap.cc:492] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: No bootstrap required, opened a new log
I20260621 06:57:55.693961 5355 ts_tablet_manager.cc:1397] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260621 06:57:55.693822 5356 ts_tablet_manager.cc:1428] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260621 06:57:55.695561 5355 raft_consensus.cc:348] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.695874 5355 raft_consensus.cc:374] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:55.695963 5355 raft_consensus.cc:729] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Initialized, Role: FOLLOWER
I20260621 06:57:55.696205 5355 consensus_queue.cc:260] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.696722 5354 consensus_queue.cc:260] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.697366 5354 ts_tablet_manager.cc:1428] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260621 06:57:55.697876 5355 ts_tablet_manager.cc:1428] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260621 06:57:55.901401 5313 raft_consensus.cc:482] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:55.901654 5313 raft_consensus.cc:504] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.902357 5313 leader_election.cc:288] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:55.902987 4927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4253ad1b244a6cb9950bce708c3730" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be" is_pre_election: true
I20260621 06:57:55.903092 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4253ad1b244a6cb9950bce708c3730" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9" is_pre_election: true
I20260621 06:57:55.903270 4927 raft_consensus.cc:2454] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:55.903422 5065 raft_consensus.cc:2454] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 0.
I20260621 06:57:55.903867 5134 leader_election.cc:302] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:55.904175 5313 raft_consensus.cc:2781] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:55.904261 5313 raft_consensus.cc:482] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:55.904408 5313 raft_consensus.cc:3037] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:55.906435 5313 raft_consensus.cc:504] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.907044 5313 leader_election.cc:288] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Requested vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429)
I20260621 06:57:55.907478 5065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4253ad1b244a6cb9950bce708c3730" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d838b90d7db454dab63db39497676b9"
I20260621 06:57:55.907486 4927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4253ad1b244a6cb9950bce708c3730" candidate_uuid: "40e49f77b7344cdcbeec0f2377a22797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be"
I20260621 06:57:55.907686 5065 raft_consensus.cc:3037] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:55.907717 4927 raft_consensus.cc:3037] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:55.910038 4927 raft_consensus.cc:2454] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:55.910079 5065 raft_consensus.cc:2454] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e49f77b7344cdcbeec0f2377a22797 in term 1.
I20260621 06:57:55.910446 5134 leader_election.cc:302] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e49f77b7344cdcbeec0f2377a22797, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:55.910779 5313 raft_consensus.cc:2781] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:55.910987 5313 raft_consensus.cc:686] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Becoming Leader. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Running, Role: LEADER
I20260621 06:57:55.911234 5313 consensus_queue.cc:237] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:55.914206 4765 catalog_manager.cc:5495] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "40e49f77b7344cdcbeec0f2377a22797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } health_report { overall_health: HEALTHY } } }
I20260621 06:57:55.992496 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:55.984041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.2.245.195:46845)
0621 06:57:55.987398 (+ 3357us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:55.987402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:55.987439 (+ 37us) server_negotiation.cc:407] Connection header received
0621 06:57:55.987504 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:55.987507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:55.987567 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:55.987718 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:55.988121 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:55.988920 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:55.990691 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:55.991108 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:55.991244 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:55.991710 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:55.991818 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:55.991926 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:55.992085 (+ 159us) server_negotiation.cc:299] Negotiation successful
0621 06:57:55.992161 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3257,"thread_start_us":3142,"threads_started":1}
I20260621 06:57:56.001466 5065 raft_consensus.cc:1261] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:56.001466 4927 raft_consensus.cc:1261] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Refusing update from remote peer 40e49f77b7344cdcbeec0f2377a22797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:56.001999 5313 consensus_queue.cc:1035] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:56.002276 5326 consensus_queue.cc:1035] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:56.049618 4765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:56.049830 4765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:56.052690 4765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Sending DeleteTablet for 3 replicas of tablet 7b4253ad1b244a6cb9950bce708c3730
I20260621 06:57:56.053612 4903 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:56 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37288
I20260621 06:57:56.053568 5042 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:56 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52410
I20260621 06:57:56.053727 5180 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4253ad1b244a6cb9950bce708c3730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:56 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33222
I20260621 06:57:56.053869 5344 tablet_replica.cc:330] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: stopping tablet replica
I20260621 06:57:56.054019 5343 tablet_replica.cc:330] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: stopping tablet replica
I20260621 06:57:56.054081 5344 raft_consensus.cc:2229] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:56.054226 5343 raft_consensus.cc:2229] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:56.054286 5344 raft_consensus.cc:2258] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:56.054464 5343 raft_consensus.cc:2258] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:56.054729 5345 tablet_replica.cc:330] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: stopping tablet replica
I20260621 06:57:56.054965 5345 raft_consensus.cc:2229] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:56.054791 4765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:57:56.055470 4765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:57:56.055274 5344 ts_tablet_manager.cc:1905] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:56.055686 5345 raft_consensus.cc:2258] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:56.056689 5343 ts_tablet_manager.cc:1905] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:56.062032 5344 ts_tablet_manager.cc:1918] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:56.062178 5344 log.cc:1196] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/7b4253ad1b244a6cb9950bce708c3730
I20260621 06:57:56.062551 5344 ts_tablet_manager.cc:1939] T 7b4253ad1b244a6cb9950bce708c3730 P 2d838b90d7db454dab63db39497676b9: Deleting consensus metadata
I20260621 06:57:56.063623 4753 catalog_manager.cc:4841] TS 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429): tablet 7b4253ad1b244a6cb9950bce708c3730 (table test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]) successfully deleted
I20260621 06:57:56.064658 5345 ts_tablet_manager.cc:1905] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:56.066794 4903 tablet_service.cc:1467] Processing CreateTablet for tablet b85809a34879450eb6ce43aa47692b64 (DEFAULT_TABLE table=test-table [id=f79c5ecfd6b944d7998745bdd8d62096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:56.066872 5042 tablet_service.cc:1467] Processing CreateTablet for tablet b85809a34879450eb6ce43aa47692b64 (DEFAULT_TABLE table=test-table [id=f79c5ecfd6b944d7998745bdd8d62096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:56.067202 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85809a34879450eb6ce43aa47692b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:56.067296 5042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85809a34879450eb6ce43aa47692b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:56.070798 5345 ts_tablet_manager.cc:1918] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:56.071386 5343 ts_tablet_manager.cc:1918] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:57:56.071574 5343 log.cc:1196] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/7b4253ad1b244a6cb9950bce708c3730
I20260621 06:57:56.071738 5356 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Bootstrap starting.
I20260621 06:57:56.072069 5343 ts_tablet_manager.cc:1939] T 7b4253ad1b244a6cb9950bce708c3730 P d379eacb6d6e45acb1d331b935a9f9be: Deleting consensus metadata
I20260621 06:57:56.071300 5180 tablet_service.cc:1467] Processing CreateTablet for tablet b85809a34879450eb6ce43aa47692b64 (DEFAULT_TABLE table=test-table [id=f79c5ecfd6b944d7998745bdd8d62096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:57:56.073330 5180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85809a34879450eb6ce43aa47692b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:57:56.073508 5355 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Bootstrap starting.
I20260621 06:57:56.073521 4752 catalog_manager.cc:4841] TS d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991): tablet 7b4253ad1b244a6cb9950bce708c3730 (table test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]) successfully deleted
I20260621 06:57:56.073624 5356 tablet_bootstrap.cc:654] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:56.075044 5355 tablet_bootstrap.cc:654] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:56.077183 5355 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: No bootstrap required, opened a new log
I20260621 06:57:56.077342 5355 ts_tablet_manager.cc:1397] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260621 06:57:56.078014 5355 raft_consensus.cc:348] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.078274 5355 raft_consensus.cc:374] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:56.078351 5355 raft_consensus.cc:729] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Initialized, Role: FOLLOWER
I20260621 06:57:56.078604 5355 consensus_queue.cc:260] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.070928 5345 log.cc:1196] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/7b4253ad1b244a6cb9950bce708c3730
I20260621 06:57:56.079362 5355 ts_tablet_manager.cc:1428] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260621 06:57:56.079435 5345 ts_tablet_manager.cc:1939] T 7b4253ad1b244a6cb9950bce708c3730 P 40e49f77b7344cdcbeec0f2377a22797: Deleting consensus metadata
I20260621 06:57:56.080657 5354 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Bootstrap starting.
I20260621 06:57:56.081971 5356 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: No bootstrap required, opened a new log
I20260621 06:57:56.082145 5354 tablet_bootstrap.cc:654] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Neither blocks nor log segments found. Creating new log.
I20260621 06:57:56.082588 5356 ts_tablet_manager.cc:1397] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260621 06:57:56.083321 4750 catalog_manager.cc:4841] TS 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845): tablet 7b4253ad1b244a6cb9950bce708c3730 (table test-table [id=e9d15ca5af3741e99c529d56ffdca2e1]) successfully deleted
I20260621 06:57:56.083987 5356 raft_consensus.cc:348] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.084311 5356 raft_consensus.cc:374] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:56.084429 5356 raft_consensus.cc:729] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d379eacb6d6e45acb1d331b935a9f9be, State: Initialized, Role: FOLLOWER
I20260621 06:57:56.084739 5356 consensus_queue.cc:260] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.085419 5356 ts_tablet_manager.cc:1428] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260621 06:57:56.086699 5354 tablet_bootstrap.cc:492] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: No bootstrap required, opened a new log
I20260621 06:57:56.086820 5354 ts_tablet_manager.cc:1397] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 06:57:56.087452 5354 raft_consensus.cc:348] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.087685 5354 raft_consensus.cc:374] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:57:56.087786 5354 raft_consensus.cc:729] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e49f77b7344cdcbeec0f2377a22797, State: Initialized, Role: FOLLOWER
I20260621 06:57:56.088076 5354 consensus_queue.cc:260] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.088652 5354 ts_tablet_manager.cc:1428] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260621 06:57:56.385813 5361 raft_consensus.cc:482] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:57:56.386035 5361 raft_consensus.cc:504] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.387609 5361 leader_election.cc:288] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845)
I20260621 06:57:56.394294 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.388083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:57823 (local address 127.2.245.195:46845)
0621 06:57:56.388303 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:56.388312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:56.388876 (+ 564us) server_negotiation.cc:407] Connection header received
0621 06:57:56.389223 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:56.389231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:56.389297 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:56.389349 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:56.390024 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.390855 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.392589 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.393366 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.393565 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.393637 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:56.393903 (+ 266us) server_negotiation.cc:299] Negotiation successful
0621 06:57:56.394026 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 06:57:56.394395 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.387914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46845 (local address 127.2.245.194:57823)
0621 06:57:56.388722 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:56.388759 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:56.389029 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:56.389500 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:56.389503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:56.389522 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:56.389802 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.389811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.391038 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.391045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.392430 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.392445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.393565 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.393571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.393731 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.393756 (+ 25us) client_negotiation.cc:769] Sending connection context
0621 06:57:56.393878 (+ 122us) client_negotiation.cc:240] Negotiation successful
0621 06:57:56.394019 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":842,"threads_started":1}
I20260621 06:57:56.395275 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.387914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36991 (local address 127.2.245.194:57079)
0621 06:57:56.389104 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:56.389125 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:56.389264 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:56.389676 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:56.389680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:56.389721 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:57:56.390108 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.390117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.391729 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.391734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.393056 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.393068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.394218 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.394224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.394610 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.394647 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 06:57:56.394741 (+ 94us) client_negotiation.cc:240] Negotiation successful
0621 06:57:56.394858 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":34,"thread_start_us":368,"threads_started":1}
I20260621 06:57:56.395278 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.388083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:57079 (local address 127.2.245.193:36991)
0621 06:57:56.389404 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:56.389408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:56.389428 (+ 20us) server_negotiation.cc:407] Connection header received
0621 06:57:56.389481 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:56.389485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:56.389534 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:56.389651 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:57:56.390288 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.391562 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.393220 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.393976 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.394219 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.394289 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:56.394787 (+ 498us) server_negotiation.cc:299] Negotiation successful
0621 06:57:56.394927 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":1014,"threads_started":1}
I20260621 06:57:56.395236 5201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85809a34879450eb6ce43aa47692b64" candidate_uuid: "2d838b90d7db454dab63db39497676b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e49f77b7344cdcbeec0f2377a22797" is_pre_election: true
I20260621 06:57:56.395659 5201 raft_consensus.cc:2454] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d838b90d7db454dab63db39497676b9 in term 0.
I20260621 06:57:56.396173 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85809a34879450eb6ce43aa47692b64" candidate_uuid: "2d838b90d7db454dab63db39497676b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be" is_pre_election: true
I20260621 06:57:56.396345 4993 leader_election.cc:302] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, 40e49f77b7344cdcbeec0f2377a22797; no voters:
I20260621 06:57:56.396421 4928 raft_consensus.cc:2454] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d838b90d7db454dab63db39497676b9 in term 0.
I20260621 06:57:56.396771 5361 raft_consensus.cc:2781] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:57:56.396911 5361 raft_consensus.cc:482] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:57:56.397047 5361 raft_consensus.cc:3037] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:56.399264 5361 raft_consensus.cc:504] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.399966 5361 leader_election.cc:288] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [CANDIDATE]: Term 1 election: Requested vote from peers d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991), 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845)
I20260621 06:57:56.400475 4928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85809a34879450eb6ce43aa47692b64" candidate_uuid: "2d838b90d7db454dab63db39497676b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d379eacb6d6e45acb1d331b935a9f9be"
I20260621 06:57:56.400496 5201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85809a34879450eb6ce43aa47692b64" candidate_uuid: "2d838b90d7db454dab63db39497676b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e49f77b7344cdcbeec0f2377a22797"
I20260621 06:57:56.400686 5201 raft_consensus.cc:3037] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:56.400686 4928 raft_consensus.cc:3037] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:57:56.404172 4928 raft_consensus.cc:2454] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d838b90d7db454dab63db39497676b9 in term 1.
I20260621 06:57:56.404172 5201 raft_consensus.cc:2454] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d838b90d7db454dab63db39497676b9 in term 1.
I20260621 06:57:56.404639 4995 leader_election.cc:302] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d838b90d7db454dab63db39497676b9, d379eacb6d6e45acb1d331b935a9f9be; no voters:
I20260621 06:57:56.404995 5361 raft_consensus.cc:2781] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:57:56.405540 5361 raft_consensus.cc:686] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 1 LEADER]: Becoming Leader. State: Replica: 2d838b90d7db454dab63db39497676b9, State: Running, Role: LEADER
I20260621 06:57:56.405958 5361 consensus_queue.cc:237] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } }
I20260621 06:57:56.409475 4766 catalog_manager.cc:5495] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d838b90d7db454dab63db39497676b9 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "2d838b90d7db454dab63db39497676b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d838b90d7db454dab63db39497676b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 } health_report { overall_health: UNKNOWN } } }
I20260621 06:57:56.458410 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.452963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.2.245.194:36429)
0621 06:57:56.453557 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:56.453561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:56.453581 (+ 20us) server_negotiation.cc:407] Connection header received
0621 06:57:56.453659 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:56.453663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:56.453744 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:56.453880 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:56.454272 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.455092 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.456379 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.456797 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.456896 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.457288 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:56.457871 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:56.457979 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:56.458084 (+ 105us) server_negotiation.cc:299] Negotiation successful
0621 06:57:56.458143 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":367,"threads_started":1}
I20260621 06:57:56.473600 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.468421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:38995 (local address 127.2.245.195:46845)
0621 06:57:56.468575 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:56.468579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:56.469148 (+ 569us) server_negotiation.cc:407] Connection header received
0621 06:57:56.469299 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:56.469303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:56.469366 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:56.469440 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:56.469939 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.470654 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.472048 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.472436 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.472532 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.472942 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:56.473042 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:56.473111 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:56.473316 (+ 205us) server_negotiation.cc:299] Negotiation successful
0621 06:57:56.473380 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 06:57:56.473626 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.468399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46845 (local address 127.2.245.194:38995)
0621 06:57:56.469059 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:56.469076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:56.469191 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:56.469469 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:56.469472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:56.469481 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:56.469789 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.469799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.470785 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.470789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.471928 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.471940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.472587 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.472591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.472711 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.472787 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:56.473155 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:56.473165 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 06:57:56.473245 (+ 80us) client_negotiation.cc:240] Negotiation successful
0621 06:57:56.473342 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":345,"threads_started":1}
I20260621 06:57:56.482003 4928 raft_consensus.cc:1261] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Refusing update from remote peer 2d838b90d7db454dab63db39497676b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:56.482360 5201 raft_consensus.cc:1261] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Refusing update from remote peer 2d838b90d7db454dab63db39497676b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:57:56.482681 5376 consensus_queue.cc:1035] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d379eacb6d6e45acb1d331b935a9f9be" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:56.483134 5361 consensus_queue.cc:1035] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e49f77b7344cdcbeec0f2377a22797" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:57:56.510097 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.504597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:45559 (local address 127.2.245.194:36429)
0621 06:57:56.504776 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:56.504785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:56.505209 (+ 424us) server_negotiation.cc:407] Connection header received
0621 06:57:56.505265 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:56.505269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:56.505374 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:56.505502 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:57:56.505950 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.506722 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.508232 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.508823 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.508948 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.509385 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:57:56.509493 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:57:56.509596 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:56.509734 (+ 138us) server_negotiation.cc:299] Negotiation successful
0621 06:57:56.509815 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 06:57:56.510335 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:56.504232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36429 (local address 127.2.245.195:45559)
0621 06:57:56.504932 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0621 06:57:56.504953 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:57:56.505094 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:57:56.505518 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:57:56.505524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:57:56.505536 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:57:56.505806 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.505814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.506840 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.506844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.508080 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:57:56.508092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:57:56.509031 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:57:56.509041 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:57:56.509180 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:56.509217 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:57:56.509595 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:57:56.509609 (+ 14us) client_negotiation.cc:769] Sending connection context
0621 06:57:56.509713 (+ 104us) client_negotiation.cc:240] Negotiation successful
0621 06:57:56.509868 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":411,"threads_started":1}
W20260621 06:57:56.511497 5327 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260621 06:57:58.109999 4766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:58.110270 4766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:57:58.114370 4766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d9b4a4a875b407fadaee2c50f5f1110: Sending DeleteTablet for 3 replicas of tablet b85809a34879450eb6ce43aa47692b64
I20260621 06:57:58.115314 4903 tablet_service.cc:1514] Processing DeleteTablet for tablet b85809a34879450eb6ce43aa47692b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:58 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37288
I20260621 06:57:58.115317 5180 tablet_service.cc:1514] Processing DeleteTablet for tablet b85809a34879450eb6ce43aa47692b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:58 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33222
I20260621 06:57:58.115368 5042 tablet_service.cc:1514] Processing DeleteTablet for tablet b85809a34879450eb6ce43aa47692b64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:57:58 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52410
I20260621 06:57:58.115942 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4837
I20260621 06:57:58.128382 5412 tablet_replica.cc:330] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: stopping tablet replica
I20260621 06:57:58.128895 5410 tablet_replica.cc:330] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: stopping tablet replica
I20260621 06:57:58.128903 5412 raft_consensus.cc:2229] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:57:58.129070 5410 raft_consensus.cc:2229] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:58.129323 5410 raft_consensus.cc:2258] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:58.129634 5412 raft_consensus.cc:2258] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:58.129899 5411 tablet_replica.cc:330] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: stopping tablet replica
I20260621 06:57:58.130112 5411 raft_consensus.cc:2229] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:57:58.130353 5411 raft_consensus.cc:2258] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:57:58.130642 5412 ts_tablet_manager.cc:1905] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:58.131417 5410 ts_tablet_manager.cc:1905] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:58.131587 5411 ts_tablet_manager.cc:1905] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:57:58.137439 5412 ts_tablet_manager.cc:1918] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:58.137599 5412 log.cc:1196] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/b85809a34879450eb6ce43aa47692b64
I20260621 06:57:58.138038 5412 ts_tablet_manager.cc:1939] T b85809a34879450eb6ce43aa47692b64 P 2d838b90d7db454dab63db39497676b9: Deleting consensus metadata
I20260621 06:57:58.138285 5411 ts_tablet_manager.cc:1918] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:58.138432 5411 log.cc:1196] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/b85809a34879450eb6ce43aa47692b64
I20260621 06:57:58.138690 5410 ts_tablet_manager.cc:1918] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 06:57:58.138823 5410 log.cc:1196] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/b85809a34879450eb6ce43aa47692b64
I20260621 06:57:58.138913 5411 ts_tablet_manager.cc:1939] T b85809a34879450eb6ce43aa47692b64 P 40e49f77b7344cdcbeec0f2377a22797: Deleting consensus metadata
I20260621 06:57:58.139147 4753 catalog_manager.cc:4841] TS 2d838b90d7db454dab63db39497676b9 (127.2.245.194:36429): tablet b85809a34879450eb6ce43aa47692b64 (table test-table [id=f79c5ecfd6b944d7998745bdd8d62096]) successfully deleted
I20260621 06:57:58.139272 5410 ts_tablet_manager.cc:1939] T b85809a34879450eb6ce43aa47692b64 P d379eacb6d6e45acb1d331b935a9f9be: Deleting consensus metadata
I20260621 06:57:58.139981 4750 catalog_manager.cc:4841] TS 40e49f77b7344cdcbeec0f2377a22797 (127.2.245.195:46845): tablet b85809a34879450eb6ce43aa47692b64 (table test-table [id=f79c5ecfd6b944d7998745bdd8d62096]) successfully deleted
I20260621 06:57:58.140275 4752 catalog_manager.cc:4841] TS d379eacb6d6e45acb1d331b935a9f9be (127.2.245.193:36991): tablet b85809a34879450eb6ce43aa47692b64 (table test-table [id=f79c5ecfd6b944d7998745bdd8d62096]) successfully deleted
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:57:58.148188 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:58.119112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.2.245.193:36991)
0621 06:57:58.124365 (+ 5253us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:58.124375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:58.128830 (+ 4455us) server_negotiation.cc:407] Connection header received
0621 06:57:58.129296 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:58.129304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:58.129485 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:58.129600 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:58.131539 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.132753 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.135120 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.135795 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.135913 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:58.143173 (+ 7260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:58.143188 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:58.143204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:58.143235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:58.145820 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.146481 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.146488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.146490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.146603 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.147014 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.147017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.147019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.147340 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:58.147565 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:58.147791 (+ 226us) server_negotiation.cc:299] Negotiation successful
0621 06:57:58.147945 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5078,"thread_start_us":4974,"threads_started":1}
W20260621 06:57:58.148794 4968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49866
W20260621 06:57:58.149437 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:58.149667 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4837
I20260621 06:57:58.167868 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4977
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:57:58.185195 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:58.168666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.2.245.194:36429)
0621 06:57:58.169178 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:58.169182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:58.169199 (+ 17us) server_negotiation.cc:407] Connection header received
0621 06:57:58.169252 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:58.169256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:58.169316 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:58.169418 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:58.171044 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.171787 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.173609 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.174028 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.174126 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:58.178983 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:58.179004 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:58.179028 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:58.179068 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:58.182230 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.182986 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.182999 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.183005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.183193 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.183596 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.183601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.183604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.183965 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:58.184266 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:58.184601 (+ 335us) server_negotiation.cc:299] Negotiation successful
0621 06:57:58.184836 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
W20260621 06:57:58.185843 5107 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34002
W20260621 06:57:58.186482 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:58.186606 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4977
I20260621 06:57:58.209174 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5116
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[4721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:57:58.224934 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:57:58.209988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.2.245.195:46845)
0621 06:57:58.210535 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0621 06:57:58.210539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:57:58.210572 (+ 33us) server_negotiation.cc:407] Connection header received
0621 06:57:58.210625 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:57:58.210628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:57:58.210691 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:57:58.210794 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:57:58.212204 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.213050 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.214356 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:57:58.214763 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:57:58.214859 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:57:58.219532 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:57:58.219547 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:57:58.219562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:57:58.219595 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:57:58.222319 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.223089 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.223096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.223098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.223218 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:57:58.223554 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:57:58.223557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:57:58.223559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:57:58.223887 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:57:58.224101 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0621 06:57:58.224449 (+ 348us) server_negotiation.cc:299] Negotiation successful
0621 06:57:58.224618 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
W20260621 06:57:58.225618 5246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44908
W20260621 06:57:58.226256 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5116: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:57:58.226445 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5116
I20260621 06:57:58.251398 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 4734
I20260621 06:57:58.252662 4806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:57:58.367483 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 4734
2026-06-21T06:57:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8895 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[5420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[5420](info): set up 2 sockets
Jun 21 06:57:58 dist-test-slave-9zdv krb5kdc[5420](info): commencing operation
krb5kdc: starting...
W20260621 06:57:59.443378 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:57:59 dist-test-slave-9zdv krb5kdc[5420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:57:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:57:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:57:59.630903 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35279
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:57:59.949671 5433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:57:59.950017 5433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:57:59.950196 5433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:57:59.957477 5433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:57:59.957628 5433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:57:59.957741 5433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:57:59.957841 5433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:57:59.957916 5433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:57:59.968132 5433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:57:59.969415 5433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:57:59.970722 5433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:57:59.982679 5438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:59.982602 5439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:57:59.982651 5441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:57:59.983019 5433 server_base.cc:1034] running on GCE node
I20260621 06:57:59.984045 5433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:57:59.985438 5433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:57:59.986653 5433 hybrid_clock.cc:648] HybridClock initialized: now 1782025079986595 us; error 53 us; skew 500 ppm
I20260621 06:57:59.987061 5433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:57:59 dist-test-slave-9zdv krb5kdc[5420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:57:59.990224 5433 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:57:59.991791 5433 webserver.cc:460] Webserver started at http://127.2.245.254:44177/ using document root <none> and password file <none>
I20260621 06:57:59.992522 5433 fs_manager.cc:359] Metadata directory not provided
I20260621 06:57:59.992676 5433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:57:59.993006 5433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:57:59.996120 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "0c633cc0bd0746eaacbcc4ce892aa29a"
format_stamp: "Formatted at 2026-06-21 06:57:59 on dist-test-slave-9zdv"
server_key: "546354d51db05ec6f411255623439764"
server_key_iv: "6ef445c388dc9fb69db50385ff054b80"
server_key_version: "encryptionkey@0"
I20260621 06:57:59.996941 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "0c633cc0bd0746eaacbcc4ce892aa29a"
format_stamp: "Formatted at 2026-06-21 06:57:59 on dist-test-slave-9zdv"
server_key: "546354d51db05ec6f411255623439764"
server_key_iv: "6ef445c388dc9fb69db50385ff054b80"
server_key_version: "encryptionkey@0"
I20260621 06:58:00.002468 5433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260621 06:58:00.005990 5448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:00.007373 5433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:00.007551 5433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "0c633cc0bd0746eaacbcc4ce892aa29a"
format_stamp: "Formatted at 2026-06-21 06:57:59 on dist-test-slave-9zdv"
server_key: "546354d51db05ec6f411255623439764"
server_key_iv: "6ef445c388dc9fb69db50385ff054b80"
server_key_version: "encryptionkey@0"
I20260621 06:58:00.007764 5433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:00.030982 5433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:00.034909 5433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:00.035274 5433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:00.053718 5433 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35279
I20260621 06:58:00.053747 5499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35279 every 8 connection(s)
I20260621 06:58:00.055132 5433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:00.058012 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5433
I20260621 06:58:00.058246 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:00.058856 3031 external_mini_cluster.cc:1428] Setting key 7e497eff379a74ecde3b0f7c0969bd4e
I20260621 06:58:00.059087 5500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:00.072863 5500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a: Bootstrap starting.
Jun 21 06:58:00 dist-test-slave-9zdv krb5kdc[5420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:00.076695 5500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:00.078087 5500 log.cc:824] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:00.081571 5500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a: No bootstrap required, opened a new log
I20260621 06:58:00.083094 5503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:00.060922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.2.245.254:35279)
0621 06:58:00.061726 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:00.061747 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:00.061802 (+ 55us) server_negotiation.cc:407] Connection header received
0621 06:58:00.063975 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:00.064021 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:00.064522 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:00.064966 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:00.066174 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.068096 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.069280 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.070356 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.070562 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:00.074408 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:00.074420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:00.074439 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:00.074495 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:00.078400 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:00.079362 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:00.079369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:00.079375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:00.079581 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:00.079962 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:00.079967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:00.079970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:00.080619 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:00.080907 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:00.081336 (+ 429us) server_negotiation.cc:299] Negotiation successful
0621 06:58:00.081757 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":318,"threads_started":1}
I20260621 06:58:00.089985 5500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } }
I20260621 06:58:00.090396 5500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:00.090473 5500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c633cc0bd0746eaacbcc4ce892aa29a, State: Initialized, Role: FOLLOWER
I20260621 06:58:00.090975 5500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } }
I20260621 06:58:00.091197 5500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:00.091295 5500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:00.091406 5500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:00.093437 5500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } }
I20260621 06:58:00.093861 5500 leader_election.cc:302] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c633cc0bd0746eaacbcc4ce892aa29a; no voters:
I20260621 06:58:00.094400 5500 leader_election.cc:288] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:00.094558 5505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:00.095372 5505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [term 1 LEADER]: Becoming Leader. State: Replica: 0c633cc0bd0746eaacbcc4ce892aa29a, State: Running, Role: LEADER
I20260621 06:58:00.095867 5505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } }
I20260621 06:58:00.096462 5500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:00.098898 5506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } } }
I20260621 06:58:00.098932 5507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c633cc0bd0746eaacbcc4ce892aa29a. Latest consensus state: current_term: 1 leader_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c633cc0bd0746eaacbcc4ce892aa29a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35279 } } }
I20260621 06:58:00.099366 5507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:00.099366 5506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:00.100415 5514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:00.106828 5514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:00.115803 5514 catalog_manager.cc:1347] Generated new cluster ID: fe664b3044844c219596f789dbd53400
I20260621 06:58:00.115928 5514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:00.128398 5514 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:00.129653 5514 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:00.135192 5514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c633cc0bd0746eaacbcc4ce892aa29a: Generated new TSK 0
I20260621 06:58:00.136034 5514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:00.191823 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:00.531427 5526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:00.531767 5526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:00.531865 5526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:00.539790 5526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:00.539911 5526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:00.540027 5526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:00.549726 5526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:00.551070 5526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:00.552352 5526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:00.565199 5532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:00.565289 5534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:00.565442 5531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:00.565826 5526 server_base.cc:1034] running on GCE node
I20260621 06:58:00.566414 5526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:00.567396 5526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:00.568627 5526 hybrid_clock.cc:648] HybridClock initialized: now 1782025080568593 us; error 48 us; skew 500 ppm
I20260621 06:58:00.568964 5526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:00 dist-test-slave-9zdv krb5kdc[5420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025080, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:00.572113 5526 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:00.573575 5526 webserver.cc:460] Webserver started at http://127.2.245.193:37105/ using document root <none> and password file <none>
I20260621 06:58:00.574301 5526 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:00.574443 5526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:00.574751 5526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:00.577895 5526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "1472f8b2e2c14ba8973a3e0c940fa874"
format_stamp: "Formatted at 2026-06-21 06:58:00 on dist-test-slave-9zdv"
server_key: "d49c6ec0921fa0f75c1b6b2f0ad47c40"
server_key_iv: "9a66296f26a3aa6a05a20f8d0a198750"
server_key_version: "encryptionkey@0"
I20260621 06:58:00.578652 5526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "1472f8b2e2c14ba8973a3e0c940fa874"
format_stamp: "Formatted at 2026-06-21 06:58:00 on dist-test-slave-9zdv"
server_key: "d49c6ec0921fa0f75c1b6b2f0ad47c40"
server_key_iv: "9a66296f26a3aa6a05a20f8d0a198750"
server_key_version: "encryptionkey@0"
I20260621 06:58:00.583731 5526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 06:58:00.587281 5541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:00.588622 5526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 06:58:00.588783 5526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "1472f8b2e2c14ba8973a3e0c940fa874"
format_stamp: "Formatted at 2026-06-21 06:58:00 on dist-test-slave-9zdv"
server_key: "d49c6ec0921fa0f75c1b6b2f0ad47c40"
server_key_iv: "9a66296f26a3aa6a05a20f8d0a198750"
server_key_version: "encryptionkey@0"
I20260621 06:58:00.588985 5526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:00.610925 5526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:00.615513 5526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:00.615823 5526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:00.616812 5526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:00.618116 5526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:00.618220 5526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:00.618397 5526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:00.618468 5526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:00.650607 5526 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36601
I20260621 06:58:00.650661 5653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36601 every 8 connection(s)
I20260621 06:58:00.651958 5526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:00.652441 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5526
I20260621 06:58:00.652657 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:00.653483 3031 external_mini_cluster.cc:1428] Setting key feb644eab8358add7631410520fe566a
Jun 21 06:58:00 dist-test-slave-9zdv krb5kdc[5420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025080, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:00.674810 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:00.654876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:53395 (local address 127.2.245.254:35279)
0621 06:58:00.655425 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:00.655432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:00.656736 (+ 1304us) server_negotiation.cc:407] Connection header received
0621 06:58:00.658523 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:00.658531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:00.658614 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:00.658735 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:00.660346 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.661064 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.663696 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.664104 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.664207 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:00.668389 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:00.668399 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:00.668406 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:00.668454 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:00.671815 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:00.672614 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:00.672620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:00.672624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:00.672793 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:00.673248 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:00.673254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:00.673259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:00.673566 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:00.673680 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:00.674394 (+ 714us) server_negotiation.cc:299] Negotiation successful
0621 06:58:00.674549 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":264,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:00.675927 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:00.655719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35279 (local address 127.2.245.193:53395)
0621 06:58:00.656547 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:00.656599 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:00.658263 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:00.658920 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:00.658928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:00.659531 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:00.660119 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:00.660148 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.661236 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.661245 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:00.663542 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:00.663560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:00.664359 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:00.664369 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:00.664685 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:00.665399 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:00.665434 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:00.668083 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:00.671954 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:00.671961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:00.671973 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:00.672450 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:00.672903 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:00.672906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:00.672908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:00.673108 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:00.673671 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:00.673677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:00.674008 (+ 331us) client_negotiation.cc:769] Sending connection context
0621 06:58:00.674288 (+ 280us) client_negotiation.cc:240] Negotiation successful
0621 06:58:00.674648 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":339,"threads_started":1}
I20260621 06:58:00.677672 5654 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35279
I20260621 06:58:00.677975 5654 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:00.678622 5654 heartbeater.cc:511] Master 127.2.245.254:35279 requested a full tablet report, sending...
I20260621 06:58:00.680985 5465 ts_manager.cc:194] Registered new tserver with Master: 1472f8b2e2c14ba8973a3e0c940fa874 (127.2.245.193:36601)
I20260621 06:58:00.682421 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:53395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:00.697996 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:01.011576 5660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:01.011858 5660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:01.011951 5660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:01.019212 5660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:01.019323 5660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:01.019434 5660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:01.028967 5660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:01.030135 5660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:01.031355 5660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:01.044143 5665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:01.044143 5666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:01.044201 5668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:01.045439 5660 server_base.cc:1034] running on GCE node
I20260621 06:58:01.046043 5660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:01.047139 5660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:01.048362 5660 hybrid_clock.cc:648] HybridClock initialized: now 1782025081048305 us; error 51 us; skew 500 ppm
I20260621 06:58:01.048671 5660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025081, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:01.052086 5660 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:01.053786 5660 webserver.cc:460] Webserver started at http://127.2.245.194:42481/ using document root <none> and password file <none>
I20260621 06:58:01.054652 5660 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:01.054793 5660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:01.055160 5660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:01.058899 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "c1bd3ec277c0407caef8c4363e758e58"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "4015c904b616f0634daa4b1704158496"
server_key_iv: "30f60bb2c767c968dfb16fda7c3e0e47"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.059926 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "c1bd3ec277c0407caef8c4363e758e58"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "4015c904b616f0634daa4b1704158496"
server_key_iv: "30f60bb2c767c968dfb16fda7c3e0e47"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.066238 5660 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260621 06:58:01.070670 5675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.072312 5660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:01.072466 5660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "c1bd3ec277c0407caef8c4363e758e58"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "4015c904b616f0634daa4b1704158496"
server_key_iv: "30f60bb2c767c968dfb16fda7c3e0e47"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.072664 5660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:01.087638 5660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:01.091786 5660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:01.092023 5660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:01.092998 5660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:01.094347 5660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:01.094420 5660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.094589 5660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:01.094666 5660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.127408 5660 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:41861
I20260621 06:58:01.127458 5787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:41861 every 8 connection(s)
I20260621 06:58:01.128671 5660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:01.135617 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5660
I20260621 06:58:01.135843 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:01.136303 3031 external_mini_cluster.cc:1428] Setting key 6a3fe32e9c3cda496780613d2e3faebc
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025081, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:01.149195 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:01.131221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:43383 (local address 127.2.245.254:35279)
0621 06:58:01.131406 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:01.131411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:01.132808 (+ 1397us) server_negotiation.cc:407] Connection header received
0621 06:58:01.134750 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:01.134755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:01.134847 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:01.134989 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:01.136817 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.137537 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.139057 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.139485 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.139614 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:01.143386 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:01.143391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:01.143395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:01.143426 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:01.145954 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:01.146873 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:01.146876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:01.146879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:01.147070 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:01.147555 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:01.147558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:01.147560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:01.147805 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:01.147927 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:01.148791 (+ 864us) server_negotiation.cc:299] Negotiation successful
0621 06:58:01.148964 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 06:58:01.150840 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:01.131711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35279 (local address 127.2.245.194:43383)
0621 06:58:01.132608 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:01.132675 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:01.134468 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:01.135240 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:01.135249 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:01.135997 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:01.136653 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:01.136670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.137704 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.137710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:01.138892 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:01.138904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.139708 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.139715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:01.139976 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:01.140678 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:01.140700 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:01.143127 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:01.146104 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:01.146116 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:01.146136 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:01.146744 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:01.147188 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:01.147195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:01.147201 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:01.147444 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:01.147945 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:01.147956 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:01.148419 (+ 463us) client_negotiation.cc:769] Sending connection context
0621 06:58:01.148711 (+ 292us) client_negotiation.cc:240] Negotiation successful
0621 06:58:01.149288 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":295,"threads_started":1}
I20260621 06:58:01.152400 5788 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35279
I20260621 06:58:01.152704 5788 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:01.153427 5788 heartbeater.cc:511] Master 127.2.245.254:35279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:58:01.154871 5465 ts_manager.cc:194] Registered new tserver with Master: c1bd3ec277c0407caef8c4363e758e58 (127.2.245.194:41861)
I20260621 06:58:01.155695 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:43383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:01.177949 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:01.490918 5793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:01.491216 5793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:01.491307 5793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:01.498526 5793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:01.498631 5793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:01.498746 5793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:01.508262 5793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:01.509492 5793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:01.510684 5793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:01.523737 5798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:01.523789 5799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:01.523708 5801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:01.528455 5793 server_base.cc:1034] running on GCE node
I20260621 06:58:01.529076 5793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:01.530048 5793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:01.531273 5793 hybrid_clock.cc:648] HybridClock initialized: now 1782025081531189 us; error 78 us; skew 500 ppm
I20260621 06:58:01.531590 5793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025081, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:01.534646 5793 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:01.536216 5793 webserver.cc:460] Webserver started at http://127.2.245.195:44687/ using document root <none> and password file <none>
I20260621 06:58:01.536947 5793 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:01.537092 5793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:01.537405 5793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:01.540597 5793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "2516dae138e245a8b9bb0bb88fd47452"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "88aeae2bd6f62ac3b0a1ed609c794281"
server_key_iv: "e1d90777ec060c6375e0eb21a10b9bfb"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.541399 5793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "2516dae138e245a8b9bb0bb88fd47452"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "88aeae2bd6f62ac3b0a1ed609c794281"
server_key_iv: "e1d90777ec060c6375e0eb21a10b9bfb"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.546464 5793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 06:58:01.549975 5808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.551307 5793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:01.551513 5793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "2516dae138e245a8b9bb0bb88fd47452"
format_stamp: "Formatted at 2026-06-21 06:58:01 on dist-test-slave-9zdv"
server_key: "88aeae2bd6f62ac3b0a1ed609c794281"
server_key_iv: "e1d90777ec060c6375e0eb21a10b9bfb"
server_key_version: "encryptionkey@0"
I20260621 06:58:01.551713 5793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:01.568670 5793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:01.572862 5793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:01.573150 5793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:01.574082 5793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:01.575590 5793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:01.575716 5793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.575862 5793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:01.575933 5793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:01.609220 5793 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:40797
I20260621 06:58:01.609264 5920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:40797 every 8 connection(s)
I20260621 06:58:01.610508 5793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:01.615254 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5793
I20260621 06:58:01.615447 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:01.615995 3031 external_mini_cluster.cc:1428] Setting key a2848401fcdc00e99a8bc74ab65368ab
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025081, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:01.630894 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:01.613311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:34005 (local address 127.2.245.254:35279)
0621 06:58:01.613504 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:01.613521 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:01.614698 (+ 1177us) server_negotiation.cc:407] Connection header received
0621 06:58:01.617055 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:01.617060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:01.617194 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:01.617297 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:01.619151 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.619898 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.621324 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.621689 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.621776 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:01.625091 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:01.625095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:01.625098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:01.625131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:01.627856 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:01.628542 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:01.628546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:01.628548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:01.628666 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:01.629061 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:01.629064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:01.629066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:01.629351 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:01.629499 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:01.630345 (+ 846us) server_negotiation.cc:299] Negotiation successful
0621 06:58:01.630596 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260621 06:58:01.631884 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:01.613835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35279 (local address 127.2.245.195:34005)
0621 06:58:01.614547 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:01.614583 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:01.616778 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:01.617581 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:01.617591 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:01.618231 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:01.618926 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:01.618969 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.620041 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.620044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:01.621204 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:01.621214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:01.621876 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:01.621881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:01.622062 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:01.622633 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:01.622657 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:01.624887 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:01.628010 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:01.628016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:01.628029 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:01.628433 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:01.628763 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:01.628766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:01.628768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:01.628933 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:01.629504 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:01.629512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:01.629924 (+ 412us) client_negotiation.cc:769] Sending connection context
0621 06:58:01.630215 (+ 291us) client_negotiation.cc:240] Negotiation successful
0621 06:58:01.630756 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":248,"threads_started":1}
I20260621 06:58:01.633124 5921 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35279
I20260621 06:58:01.633385 5921 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:01.634019 5921 heartbeater.cc:511] Master 127.2.245.254:35279 requested a full tablet report, sending...
I20260621 06:58:01.635313 5465 ts_manager.cc:194] Registered new tserver with Master: 2516dae138e245a8b9bb0bb88fd47452 (127.2.245.195:40797)
I20260621 06:58:01.636050 5465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:34005
I20260621 06:58:01.643280 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 06:58:01.673502 5656 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:01.670978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.2.245.254:35279)
0621 06:58:01.671176 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:01.671184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:01.671588 (+ 404us) server_negotiation.cc:407] Connection header received
0621 06:58:01.671791 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:01.671800 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:01.671864 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:01.671974 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:01.673215 (+ 1241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46914: BlockingRecv error: recv got EOF from 127.0.0.1:46914 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:01.676656 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5526
W20260621 06:58:01.679420 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5526: Not authorized: Client connection negotiation failed: client connection to 127.2.245.193:36601: server requires authentication, but client does not have Kerberos credentials available
I20260621 06:58:01.679589 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5526
I20260621 06:58:01.697609 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5660
W20260621 06:58:01.700282 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5660: Not authorized: Client connection negotiation failed: client connection to 127.2.245.194:41861: server requires authentication, but client does not have Kerberos credentials available
I20260621 06:58:01.700444 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5660
I20260621 06:58:01.715869 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5793
W20260621 06:58:01.718843 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5793: Not authorized: Client connection negotiation failed: client connection to 127.2.245.195:40797: server requires authentication, but client does not have Kerberos credentials available
I20260621 06:58:01.719023 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5793
I20260621 06:58:01.735351 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5433
I20260621 06:58:01.736585 5495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:01.836786 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5433
2026-06-21T06:58:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3466 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5938](info): set up 2 sockets
Jun 21 06:58:01 dist-test-slave-9zdv krb5kdc[5938](info): commencing operation
krb5kdc: starting...
W20260621 06:58:02.927507 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:03.117830 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44131
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:03.436200 5951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:03.436482 5951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:03.436571 5951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:03.443598 5951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:03.443703 5951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:03.443748 5951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:03.443804 5951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:03.443851 5951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:03.453593 5951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:03.454914 5951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:03.456198 5951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:03.467980 5956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:03.468021 5957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:03.468091 5959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:03.468894 5951 server_base.cc:1034] running on GCE node
I20260621 06:58:03.469655 5951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:03.471076 5951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:03.472302 5951 hybrid_clock.cc:648] HybridClock initialized: now 1782025083472260 us; error 36 us; skew 500 ppm
I20260621 06:58:03.472679 5951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:03.478914 5951 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:03.480504 5951 webserver.cc:460] Webserver started at http://127.2.245.254:39119/ using document root <none> and password file <none>
I20260621 06:58:03.481181 5951 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:03.481336 5951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:03.481639 5951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:03.484671 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "ee518e8fc3134e67910b30ed430f5d33"
format_stamp: "Formatted at 2026-06-21 06:58:03 on dist-test-slave-9zdv"
server_key: "7ddb9cec309ad564d2f31f4f4c4a9e5f"
server_key_iv: "0d25a1115b616e44798c05101c577579"
server_key_version: "encryptionkey@0"
I20260621 06:58:03.485500 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "ee518e8fc3134e67910b30ed430f5d33"
format_stamp: "Formatted at 2026-06-21 06:58:03 on dist-test-slave-9zdv"
server_key: "7ddb9cec309ad564d2f31f4f4c4a9e5f"
server_key_iv: "0d25a1115b616e44798c05101c577579"
server_key_version: "encryptionkey@0"
I20260621 06:58:03.490692 5951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:58:03.494138 5966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:03.495623 5951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:03.495807 5951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "ee518e8fc3134e67910b30ed430f5d33"
format_stamp: "Formatted at 2026-06-21 06:58:03 on dist-test-slave-9zdv"
server_key: "7ddb9cec309ad564d2f31f4f4c4a9e5f"
server_key_iv: "0d25a1115b616e44798c05101c577579"
server_key_version: "encryptionkey@0"
I20260621 06:58:03.496004 5951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:03.521843 5951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:03.526386 5951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:03.526682 5951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:03.545288 5951 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:44131
I20260621 06:58:03.545310 6017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:44131 every 8 connection(s)
I20260621 06:58:03.546799 5951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:03.550498 6018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:03.556161 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 5951
I20260621 06:58:03.556370 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:03.557027 3031 external_mini_cluster.cc:1428] Setting key 57f1b6c61ab0ff4ef8d935656660b475
I20260621 06:58:03.561192 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33: Bootstrap starting.
I20260621 06:58:03.564927 6018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:03.566083 6018 log.cc:824] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:03.569401 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33: No bootstrap required, opened a new log
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:03.579928 6018 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } }
I20260621 06:58:03.580499 6018 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:03.580631 6018 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee518e8fc3134e67910b30ed430f5d33, State: Initialized, Role: FOLLOWER
I20260621 06:58:03.581302 6018 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } }
I20260621 06:58:03.581612 6018 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:03.581669 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:03.558902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.2.245.254:44131)
0621 06:58:03.559767 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:03.559781 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:03.559820 (+ 39us) server_negotiation.cc:407] Connection header received
0621 06:58:03.561462 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:03.561485 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:03.561853 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:03.562165 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:03.564051 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:03.565501 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:03.567224 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:03.567902 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:03.568079 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:03.572903 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:03.572914 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:03.572934 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:03.572990 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:03.577062 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:03.577708 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:03.577717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:03.577726 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:03.577931 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:03.578285 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:03.578290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:03.578294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:03.579065 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:03.579370 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:03.579721 (+ 351us) server_negotiation.cc:299] Negotiation successful
0621 06:58:03.580240 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":364,"threads_started":1}
I20260621 06:58:03.581776 6018 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:03.582001 6018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:03.584980 6018 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } }
I20260621 06:58:03.585568 6018 leader_election.cc:302] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee518e8fc3134e67910b30ed430f5d33; no voters:
I20260621 06:58:03.585755 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 06:58:03.586207 6018 leader_election.cc:288] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:03.586310 6023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:03.586973 6023 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [term 1 LEADER]: Becoming Leader. State: Replica: ee518e8fc3134e67910b30ed430f5d33, State: Running, Role: LEADER
I20260621 06:58:03.587572 6023 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } }
I20260621 06:58:03.589268 6023 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee518e8fc3134e67910b30ed430f5d33. Latest consensus state: current_term: 1 leader_uuid: "ee518e8fc3134e67910b30ed430f5d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } } }
I20260621 06:58:03.589627 6023 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:03.590045 6025 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee518e8fc3134e67910b30ed430f5d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee518e8fc3134e67910b30ed430f5d33" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44131 } } }
I20260621 06:58:03.590404 6025 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:03.601403 6018 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:03.601629 6027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:03.606657 6027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:03.619213 6027 catalog_manager.cc:1347] Generated new cluster ID: 8e7aa844caa8486598c61250379a064e
I20260621 06:58:03.619321 6027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:03.639657 6027 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:03.641554 6027 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:03.654009 6027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee518e8fc3134e67910b30ed430f5d33: Generated new TSK 0
I20260621 06:58:03.655229 6027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:58:03.655638 6045 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 06:58:03.666124 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:03.657816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.2.245.254:44131)
0621 06:58:03.657991 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:03.657996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:03.658526 (+ 530us) server_negotiation.cc:407] Connection header received
0621 06:58:03.658662 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:03.658669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:03.658747 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:03.658905 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:03.659923 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:03.661034 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:03.662663 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:03.663330 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:03.663455 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:03.664473 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:03.664479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:03.664484 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:03.665324 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:03.665396 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:03.665424 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:03.665535 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:03.665666 (+ 131us) server_negotiation.cc:299] Negotiation successful
0621 06:58:03.665767 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 06:58:03.673096 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 5951
I20260621 06:58:03.674400 6013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:03.767478 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 5951
2026-06-21T06:58:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1925 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[6052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[6052](info): set up 2 sockets
Jun 21 06:58:03 dist-test-slave-9zdv krb5kdc[6052](info): commencing operation
krb5kdc: starting...
W20260621 06:58:04.855528 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.005s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:04 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:05.042313 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34303
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:05.356842 6065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:05.357125 6065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:05.357293 6065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:05.364562 6065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:05.364676 6065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:05.364815 6065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:05.364918 6065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:05.365001 6065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:05.374792 6065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:05.376163 6065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:05.377489 6065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:05.389099 6071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:05.389158 6070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:05.389202 6073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:05.389837 6065 server_base.cc:1034] running on GCE node
I20260621 06:58:05.390548 6065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:05.391964 6065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:05.393213 6065 hybrid_clock.cc:648] HybridClock initialized: now 1782025085393156 us; error 42 us; skew 500 ppm
I20260621 06:58:05.393534 6065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:05 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025085, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:05.396891 6065 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:05.398335 6065 webserver.cc:460] Webserver started at http://127.2.245.254:36191/ using document root <none> and password file <none>
I20260621 06:58:05.399070 6065 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:05.399178 6065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:05.399426 6065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:05.402310 6065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "794ba1eff60f4fae84953d42130237e0"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b9fec00f3d1d0e506c38949e5d2a4d6b"
server_key_iv: "baa48f7fbd8b40b837b118ea4adeb6fb"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.403105 6065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "794ba1eff60f4fae84953d42130237e0"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b9fec00f3d1d0e506c38949e5d2a4d6b"
server_key_iv: "baa48f7fbd8b40b837b118ea4adeb6fb"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.408326 6065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:58:05.412068 6080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:05.413458 6065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:58:05.413637 6065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "794ba1eff60f4fae84953d42130237e0"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b9fec00f3d1d0e506c38949e5d2a4d6b"
server_key_iv: "baa48f7fbd8b40b837b118ea4adeb6fb"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.413811 6065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:05.434149 6065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:05.438207 6065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:05.438491 6065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:05.456333 6065 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:34303
I20260621 06:58:05.456362 6131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:34303 every 8 connection(s)
I20260621 06:58:05.457664 6065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:05.459968 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6065
I20260621 06:58:05.460173 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:05.460815 3031 external_mini_cluster.cc:1428] Setting key 93d4ea251737247a4612beb477006741
I20260621 06:58:05.461437 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:05.475837 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0: Bootstrap starting.
Jun 21 06:58:05 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:05.479511 6132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:05.480564 6132 log.cc:824] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:05.483768 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0: No bootstrap required, opened a new log
I20260621 06:58:05.484650 6135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:05.462710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.2.245.254:34303)
0621 06:58:05.463577 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:05.463589 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:05.463629 (+ 40us) server_negotiation.cc:407] Connection header received
0621 06:58:05.465908 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:05.465930 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:05.466330 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:05.466652 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:05.468457 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:05.469717 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:05.471661 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:05.472368 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:05.472506 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:05.477302 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:05.477315 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:05.477348 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:05.477412 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:05.480649 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:05.481471 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:05.481482 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:05.481493 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:05.481706 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:05.482103 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:05.482106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:05.482108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:05.482593 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:05.482807 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:05.483260 (+ 453us) server_negotiation.cc:299] Negotiation successful
0621 06:58:05.483628 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":437,"threads_started":1}
I20260621 06:58:05.491068 6132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } }
I20260621 06:58:05.491427 6132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:05.491528 6132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794ba1eff60f4fae84953d42130237e0, State: Initialized, Role: FOLLOWER
I20260621 06:58:05.492152 6132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } }
I20260621 06:58:05.492393 6132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:05.492550 6132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:05.492733 6132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:05.494891 6132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } }
I20260621 06:58:05.495442 6132 leader_election.cc:302] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 794ba1eff60f4fae84953d42130237e0; no voters:
I20260621 06:58:05.495997 6132 leader_election.cc:288] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:05.496125 6137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:05.496821 6137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [term 1 LEADER]: Becoming Leader. State: Replica: 794ba1eff60f4fae84953d42130237e0, State: Running, Role: LEADER
I20260621 06:58:05.497371 6137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } }
I20260621 06:58:05.498113 6132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:05.500164 6139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794ba1eff60f4fae84953d42130237e0. Latest consensus state: current_term: 1 leader_uuid: "794ba1eff60f4fae84953d42130237e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } } }
I20260621 06:58:05.500584 6139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:05.502909 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "794ba1eff60f4fae84953d42130237e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794ba1eff60f4fae84953d42130237e0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34303 } } }
I20260621 06:58:05.503324 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:05.503443 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:05.508837 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:05.517968 6142 catalog_manager.cc:1347] Generated new cluster ID: 099996b9ef714f74927e6a21b486959c
I20260621 06:58:05.518118 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:05.534046 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:05.535586 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:05.543869 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 794ba1eff60f4fae84953d42130237e0: Generated new TSK 0
I20260621 06:58:05.544643 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:05.593874 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:05.934828 6158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:05.935145 6158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:05.935240 6158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:05.942710 6158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:05.942813 6158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:05.942924 6158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:05.952483 6158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:05.953809 6158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:05.955108 6158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:05.968106 6163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:05.968165 6164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:05.968204 6166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:05.969192 6158 server_base.cc:1034] running on GCE node
I20260621 06:58:05.969869 6158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:05.970760 6158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:05.971953 6158 hybrid_clock.cc:648] HybridClock initialized: now 1782025085971892 us; error 51 us; skew 500 ppm
I20260621 06:58:05.972275 6158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:05 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025085, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:05.975381 6158 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:05.976820 6158 webserver.cc:460] Webserver started at http://127.2.245.193:45515/ using document root <none> and password file <none>
I20260621 06:58:05.977456 6158 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:05.977552 6158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:05.977797 6158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:05.980607 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "4ae16cae8e5443c3a4468df23ce1a01d"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b587635c92d208bef52ec742ece1d586"
server_key_iv: "a8fb78f4fcd48b5163fa10a8ddd2ae48"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.981308 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "4ae16cae8e5443c3a4468df23ce1a01d"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b587635c92d208bef52ec742ece1d586"
server_key_iv: "a8fb78f4fcd48b5163fa10a8ddd2ae48"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.986492 6158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260621 06:58:05.989856 6173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:05.991292 6158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 06:58:05.991446 6158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "4ae16cae8e5443c3a4468df23ce1a01d"
format_stamp: "Formatted at 2026-06-21 06:58:05 on dist-test-slave-9zdv"
server_key: "b587635c92d208bef52ec742ece1d586"
server_key_iv: "a8fb78f4fcd48b5163fa10a8ddd2ae48"
server_key_version: "encryptionkey@0"
I20260621 06:58:05.991623 6158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:06.010527 6158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:06.014448 6158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:06.014712 6158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:06.015663 6158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:06.016914 6158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:06.016983 6158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.017064 6158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:06.017107 6158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.049695 6158 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:35261
I20260621 06:58:06.049783 6285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:35261 every 8 connection(s)
I20260621 06:58:06.050930 6158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:06.054874 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6158
I20260621 06:58:06.055096 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:06.055730 3031 external_mini_cluster.cc:1428] Setting key 9fad4976b8f82294df04ed68c6cbffac
Jun 21 06:58:06 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025085, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:06.073769 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:06.053573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:37705 (local address 127.2.245.254:34303)
0621 06:58:06.054011 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:06.054016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:06.054883 (+ 867us) server_negotiation.cc:407] Connection header received
0621 06:58:06.056952 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:06.056958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:06.057051 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:06.057170 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:06.059212 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.059917 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.062478 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.062876 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.063079 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:06.066864 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:06.066870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:06.066877 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:06.066923 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:06.070509 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:06.071301 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:06.071309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:06.071316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:06.071478 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:06.071889 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:06.071897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:06.071903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:06.072310 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:06.072483 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:06.073344 (+ 861us) server_negotiation.cc:299] Negotiation successful
0621 06:58:06.073516 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":240,"threads_started":1}
I20260621 06:58:06.075500 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:06.053965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34303 (local address 127.2.245.193:37705)
0621 06:58:06.054722 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:06.054765 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:06.056633 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:06.057412 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:06.057420 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:06.058130 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:06.058988 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:06.059018 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.060083 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.060092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:06.062319 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:06.062337 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.063202 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.063211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:06.063523 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:06.064216 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:06.064242 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:06.066633 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:06.070649 (+ 4016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:06.070656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:06.070669 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:06.071178 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:06.071589 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:06.071592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:06.071594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:06.071766 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:06.072436 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:06.072441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:06.072850 (+ 409us) client_negotiation.cc:769] Sending connection context
0621 06:58:06.073253 (+ 403us) client_negotiation.cc:240] Negotiation successful
0621 06:58:06.073897 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":274,"threads_started":1}
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:06.077015 6286 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34303
I20260621 06:58:06.077313 6286 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:06.078101 6286 heartbeater.cc:511] Master 127.2.245.254:34303 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:06.080216 6097 ts_manager.cc:194] Registered new tserver with Master: 4ae16cae8e5443c3a4468df23ce1a01d (127.2.245.193:35261)
I20260621 06:58:06.081581 6097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:37705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:06.101598 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:06.420090 6292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:06.420362 6292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:06.420454 6292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:06.427528 6292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:06.427631 6292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:06.427742 6292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:06.436964 6292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:06.438427 6292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:06.439656 6292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:06.452334 6298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:06.452577 6297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:06.452924 6300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:06.453202 6292 server_base.cc:1034] running on GCE node
I20260621 06:58:06.453780 6292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:06.454682 6292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:06.455946 6292 hybrid_clock.cc:648] HybridClock initialized: now 1782025086455858 us; error 79 us; skew 500 ppm
I20260621 06:58:06.456305 6292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:06 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025086, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:06.460057 6292 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:06.471421 6292 webserver.cc:460] Webserver started at http://127.2.245.194:39985/ using document root <none> and password file <none>
I20260621 06:58:06.472121 6292 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:06.472265 6292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:06.472570 6292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:06.475543 6292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "29b30b9b35174e5fa4bb45729a48d251"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "4fe6635f89c9411cdf7811a25c0476a4"
server_key_iv: "9b4007afe6cee48c1b7c35aa7b1b3e35"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.476327 6292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "29b30b9b35174e5fa4bb45729a48d251"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "4fe6635f89c9411cdf7811a25c0476a4"
server_key_iv: "9b4007afe6cee48c1b7c35aa7b1b3e35"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.481412 6292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:06.484786 6307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.486021 6292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 06:58:06.486199 6292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "29b30b9b35174e5fa4bb45729a48d251"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "4fe6635f89c9411cdf7811a25c0476a4"
server_key_iv: "9b4007afe6cee48c1b7c35aa7b1b3e35"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.486389 6292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:06.501754 6292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:06.506089 6292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:06.506381 6292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:06.507367 6292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:06.508673 6292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:06.508767 6292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.508915 6292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:06.508991 6292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.540745 6292 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42167
I20260621 06:58:06.540782 6419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42167 every 8 connection(s)
I20260621 06:58:06.542140 6292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:06.549263 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6292
I20260621 06:58:06.549443 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:06.549873 3031 external_mini_cluster.cc:1428] Setting key 65cc4975a3e36b36f5523b88762e5c8e
Jun 21 06:58:06 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025086, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:06.562033 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:06.544598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:55861 (local address 127.2.245.254:34303)
0621 06:58:06.544763 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:06.544767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:06.545946 (+ 1179us) server_negotiation.cc:407] Connection header received
0621 06:58:06.547865 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:06.547870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:06.547979 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:06.548115 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:06.549662 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.550452 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.552551 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.553032 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.553174 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:06.556633 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:06.556638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:06.556641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:06.556671 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:06.559195 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:06.559899 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:06.559903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:06.559905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:06.560027 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:06.560454 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:06.560458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:06.560460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:06.560704 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:06.560855 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:06.561515 (+ 660us) server_negotiation.cc:299] Negotiation successful
0621 06:58:06.561721 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:06.563105 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:06.545070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34303 (local address 127.2.245.194:55861)
0621 06:58:06.545776 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:06.545819 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:06.547576 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:06.548355 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:06.548363 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:06.548829 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:06.549467 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:06.549506 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.550603 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.550609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:06.552362 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:06.552408 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:06.553261 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:06.553268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:06.553511 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:06.554201 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:06.554222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:06.556400 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:06.559336 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:06.559344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:06.559359 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:06.559758 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:06.560144 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:06.560149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:06.560152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:06.560329 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:06.560830 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:06.560838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:06.561160 (+ 322us) client_negotiation.cc:769] Sending connection context
0621 06:58:06.561419 (+ 259us) client_negotiation.cc:240] Negotiation successful
0621 06:58:06.561881 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":265,"threads_started":1}
I20260621 06:58:06.564563 6420 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34303
I20260621 06:58:06.564886 6420 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:06.565796 6420 heartbeater.cc:511] Master 127.2.245.254:34303 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:58:06.567471 6097 ts_manager.cc:194] Registered new tserver with Master: 29b30b9b35174e5fa4bb45729a48d251 (127.2.245.194:42167)
I20260621 06:58:06.568347 6097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:55861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:06.589936 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:06.900446 6425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:06.900725 6425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:06.900878 6425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:06.908270 6425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:06.908393 6425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:06.908564 6425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:06.918072 6425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:06.919411 6425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:06.920635 6425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:06.933303 6431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:06.933337 6433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:06.933373 6430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:06.934019 6425 server_base.cc:1034] running on GCE node
I20260621 06:58:06.934602 6425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:06.935643 6425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:06.936832 6425 hybrid_clock.cc:648] HybridClock initialized: now 1782025086936783 us; error 46 us; skew 500 ppm
I20260621 06:58:06.937144 6425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:06 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025086, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:06.940662 6425 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:06.948084 6425 webserver.cc:460] Webserver started at http://127.2.245.195:44835/ using document root <none> and password file <none>
I20260621 06:58:06.948997 6425 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:06.949132 6425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:06.949424 6425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:06.952394 6425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "fd76d9d12534441990a83c07b8d6d05d"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "2ea319f345e988623b003f11d2060f40"
server_key_iv: "d5c16498d55a7110989ec681242c3bbb"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.953176 6425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "fd76d9d12534441990a83c07b8d6d05d"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "2ea319f345e988623b003f11d2060f40"
server_key_iv: "d5c16498d55a7110989ec681242c3bbb"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.958200 6425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:58:06.961573 6440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.962858 6425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:06.963055 6425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "fd76d9d12534441990a83c07b8d6d05d"
format_stamp: "Formatted at 2026-06-21 06:58:06 on dist-test-slave-9zdv"
server_key: "2ea319f345e988623b003f11d2060f40"
server_key_iv: "d5c16498d55a7110989ec681242c3bbb"
server_key_version: "encryptionkey@0"
I20260621 06:58:06.963339 6425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:06.988533 6425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:06.993029 6425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:06.993321 6425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:06.994279 6425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:06.995750 6425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:06.995824 6425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:06.995996 6425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:06.996045 6425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:07.027632 6425 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:33429
I20260621 06:58:07.027666 6552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:33429 every 8 connection(s)
I20260621 06:58:07.028864 6425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:07.036616 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6425
I20260621 06:58:07.036790 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:07.037356 3031 external_mini_cluster.cc:1428] Setting key 048933d96fc3a248112a153bf82c256a
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025086, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:07.048224 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.031322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:41069 (local address 127.2.245.254:34303)
0621 06:58:07.031502 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:07.031506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:07.032505 (+ 999us) server_negotiation.cc:407] Connection header received
0621 06:58:07.034310 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:07.034315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:07.034426 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:07.034551 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:07.035995 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.037212 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.039239 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.039698 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.039789 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.043065 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:07.043070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:07.043073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:07.043101 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:07.045637 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.046246 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.046250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.046253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.046371 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.046734 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.046736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.046738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.047046 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:07.047145 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:07.047845 (+ 700us) server_negotiation.cc:299] Negotiation successful
0621 06:58:07.047989 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 06:58:07.049271 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.031667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34303 (local address 127.2.245.195:41069)
0621 06:58:07.032331 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:07.032373 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:07.034025 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:07.034767 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:07.034774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:07.035287 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:07.035822 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:07.035838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.037370 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.037374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:07.039103 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:07.039118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.039915 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.039923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:07.040219 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.040835 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:07.040854 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:07.042836 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:07.045752 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:07.045758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:07.045769 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:07.046149 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:07.046461 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:07.046464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:07.046466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:07.046647 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:07.047167 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:07.047176 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:07.047552 (+ 376us) client_negotiation.cc:769] Sending connection context
0621 06:58:07.047765 (+ 213us) client_negotiation.cc:240] Negotiation successful
0621 06:58:07.048104 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":261,"threads_started":1}
I20260621 06:58:07.050660 6553 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34303
I20260621 06:58:07.051043 6553 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:07.051900 6553 heartbeater.cc:511] Master 127.2.245.254:34303 requested a full tablet report, sending...
I20260621 06:58:07.053387 6097 ts_manager.cc:194] Registered new tserver with Master: fd76d9d12534441990a83c07b8d6d05d (127.2.245.195:33429)
I20260621 06:58:07.054175 6097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:41069
I20260621 06:58:07.064730 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:07.084738 6286 heartbeater.cc:503] Master 127.2.245.254:34303 was elected leader, sending a full tablet report...
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:07.112367 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.098509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.2.245.254:34303)
0621 06:58:07.098725 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:07.098735 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:07.099249 (+ 514us) server_negotiation.cc:407] Connection header received
0621 06:58:07.099388 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:07.099392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:07.099479 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:07.099561 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:07.100791 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.101519 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.102838 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.103328 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.103478 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.107363 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:07.107368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:07.107371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:07.107402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:07.109714 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.110389 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.110394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.110398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.110593 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.111067 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.111075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.111081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.111428 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:07.111616 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:07.111887 (+ 271us) server_negotiation.cc:299] Negotiation successful
0621 06:58:07.112039 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260621 06:58:07.112885 6097 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47504
I20260621 06:58:07.115100 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6158
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:07.130015 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.116098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.2.245.193:35261)
0621 06:58:07.116695 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:07.116702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:07.116726 (+ 24us) server_negotiation.cc:407] Connection header received
0621 06:58:07.116777 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:07.116783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:07.116990 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:07.117175 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:07.118346 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.119480 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.120574 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.121117 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.121208 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.124733 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:07.124744 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:07.124757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:07.124786 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:07.127656 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.128223 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.128232 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.128235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.128348 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.128664 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.128667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.128669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.129025 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:07.129240 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:07.129427 (+ 187us) server_negotiation.cc:299] Negotiation successful
0621 06:58:07.129612 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":314,"threads_started":1}
W20260621 06:58:07.131223 6281 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51108
W20260621 06:58:07.132179 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6158: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:58:07.132340 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6158
I20260621 06:58:07.148087 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6292
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:58:07.163314 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.148899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36420 (local address 127.2.245.194:42167)
0621 06:58:07.149507 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:07.149514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:07.149539 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:58:07.149597 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:07.149604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:07.149823 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:07.149950 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:07.151433 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.152468 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.153668 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.154210 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.154305 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.158175 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:07.158186 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:07.158200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:07.158235 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:07.161025 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.161559 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.161566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.161569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.161683 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.162019 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.162022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.162024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.162332 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:07.162536 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:07.162771 (+ 235us) server_negotiation.cc:299] Negotiation successful
0621 06:58:07.163037 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":332,"threads_started":1}
W20260621 06:58:07.164598 6415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36420
W20260621 06:58:07.165675 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:58:07.165798 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6292
I20260621 06:58:07.180864 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6425
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:58:07.196947 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:07.181857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.2.245.195:33429)
0621 06:58:07.182522 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:07.182528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:07.182556 (+ 28us) server_negotiation.cc:407] Connection header received
0621 06:58:07.182619 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:07.182626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:07.182828 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:07.183034 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:07.184261 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.185270 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.186418 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:07.187101 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:07.187194 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:07.191342 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:07.191354 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:07.191367 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:07.191401 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:07.194465 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.195121 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.195128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.195130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.195241 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:07.195565 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:07.195568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:07.195570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:07.195886 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:07.196110 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:07.196356 (+ 246us) server_negotiation.cc:299] Negotiation successful
0621 06:58:07.196564 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":371,"threads_started":1}
W20260621 06:58:07.198282 6548 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34116
W20260621 06:58:07.199432 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6425: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 06:58:07.199579 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6425
I20260621 06:58:07.215088 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6065
I20260621 06:58:07.216192 6127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:07.317340 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6065
2026-06-21T06:58:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3555 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6570](info): set up 2 sockets
Jun 21 06:58:07 dist-test-slave-9zdv krb5kdc[6570](info): commencing operation
krb5kdc: starting...
W20260621 06:58:08.387440 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:08 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:08.576112 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44071
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:08.898905 6583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:08.899256 6583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:08.899358 6583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:08.906584 6583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:08.906692 6583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:08.906737 6583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:08.906785 6583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:08.906829 6583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:08.916725 6583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:08.917944 6583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:08.919239 6583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:08.931488 6589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:08.931630 6591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:08.931854 6588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:08.932483 6583 server_base.cc:1034] running on GCE node
I20260621 06:58:08.933213 6583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:08.934592 6583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:08.935837 6583 hybrid_clock.cc:648] HybridClock initialized: now 1782025088935789 us; error 41 us; skew 500 ppm
I20260621 06:58:08.936198 6583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:08 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:08.939357 6583 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:08.940877 6583 webserver.cc:460] Webserver started at http://127.2.245.254:36825/ using document root <none> and password file <none>
I20260621 06:58:08.941533 6583 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:08.941634 6583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:08.941905 6583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:08.945058 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "6d2c1db0274e4f98b6f6cb1d59165931"
format_stamp: "Formatted at 2026-06-21 06:58:08 on dist-test-slave-9zdv"
server_key: "0596dd3c1b55a376be2fdc7fdec689e1"
server_key_iv: "3401a6f34581abdcf2f5923a744f72f0"
server_key_version: "encryptionkey@0"
I20260621 06:58:08.945806 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "6d2c1db0274e4f98b6f6cb1d59165931"
format_stamp: "Formatted at 2026-06-21 06:58:08 on dist-test-slave-9zdv"
server_key: "0596dd3c1b55a376be2fdc7fdec689e1"
server_key_iv: "3401a6f34581abdcf2f5923a744f72f0"
server_key_version: "encryptionkey@0"
I20260621 06:58:08.951009 6583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:58:08.954336 6598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:08.955688 6583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 06:58:08.955819 6583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "6d2c1db0274e4f98b6f6cb1d59165931"
format_stamp: "Formatted at 2026-06-21 06:58:08 on dist-test-slave-9zdv"
server_key: "0596dd3c1b55a376be2fdc7fdec689e1"
server_key_iv: "3401a6f34581abdcf2f5923a744f72f0"
server_key_version: "encryptionkey@0"
I20260621 06:58:08.955974 6583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:08.972935 6583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:08.976800 6583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:08.977030 6583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:08.995616 6583 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:44071
I20260621 06:58:08.995661 6649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:44071 every 8 connection(s)
I20260621 06:58:08.996974 6583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:09.000624 6650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:09.003176 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6583
I20260621 06:58:09.003407 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:09.004000 3031 external_mini_cluster.cc:1428] Setting key 2fbcf716317f895c9405f655f4eca3cb
I20260621 06:58:09.011669 6650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931: Bootstrap starting.
I20260621 06:58:09.014616 6650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:09.015715 6650 log.cc:824] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931: Log is configured to *not* fsync() on all Append() calls
Jun 21 06:58:09 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:09.018538 6650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931: No bootstrap required, opened a new log
I20260621 06:58:09.025607 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:09.005984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.2.245.254:44071)
0621 06:58:09.006731 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:09.006742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:09.006780 (+ 38us) server_negotiation.cc:407] Connection header received
0621 06:58:09.008449 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:09.008470 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:09.008852 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:09.009247 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:09.010686 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.012030 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.013292 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.013928 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.014122 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:09.018491 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:09.018498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:09.018511 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:09.018545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:09.021654 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:09.022399 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:09.022405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:09.022410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:09.022570 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:09.022932 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:09.022962 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:09.022967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:09.023444 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:09.023774 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:09.024190 (+ 416us) server_negotiation.cc:299] Negotiation successful
0621 06:58:09.024617 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":343,"threads_started":1}
I20260621 06:58:09.026446 6650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } }
I20260621 06:58:09.026813 6650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:09.026952 6650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d2c1db0274e4f98b6f6cb1d59165931, State: Initialized, Role: FOLLOWER
I20260621 06:58:09.027563 6650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } }
I20260621 06:58:09.027828 6650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:09.027976 6650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:09.028151 6650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:09.030282 6650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } }
I20260621 06:58:09.030772 6650 leader_election.cc:302] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d2c1db0274e4f98b6f6cb1d59165931; no voters:
I20260621 06:58:09.031489 6650 leader_election.cc:288] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:09.031656 6655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:09.032995 6655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [term 1 LEADER]: Becoming Leader. State: Replica: 6d2c1db0274e4f98b6f6cb1d59165931, State: Running, Role: LEADER
I20260621 06:58:09.033524 6655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } }
I20260621 06:58:09.033636 6650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:09.036324 6656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } } }
I20260621 06:58:09.036355 6657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d2c1db0274e4f98b6f6cb1d59165931. Latest consensus state: current_term: 1 leader_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d2c1db0274e4f98b6f6cb1d59165931" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44071 } } }
I20260621 06:58:09.036751 6657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:09.036751 6656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:09.037612 6662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:09.042831 6662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:09.051407 6662 catalog_manager.cc:1347] Generated new cluster ID: fdf162182d8d4bf587e393a32936c52d
I20260621 06:58:09.051533 6662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:09.068238 6662 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:09.069422 6662 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:09.079206 6662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d2c1db0274e4f98b6f6cb1d59165931: Generated new TSK 0
I20260621 06:58:09.079984 6662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:09.130903 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:09.476362 6676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:09.476630 6676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:09.476804 6676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:09.484149 6676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:09.484275 6676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:09.484462 6676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:09.494004 6676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:09.495416 6676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:09.496703 6676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:09.509397 6681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:09.509456 6682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:09.509552 6684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:09.510061 6676 server_base.cc:1034] running on GCE node
I20260621 06:58:09.510618 6676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:09.511581 6676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:09.512805 6676 hybrid_clock.cc:648] HybridClock initialized: now 1782025089512765 us; error 36 us; skew 500 ppm
I20260621 06:58:09.513125 6676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:09 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:09.516691 6676 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:09.528162 6676 webserver.cc:460] Webserver started at http://127.2.245.193:41681/ using document root <none> and password file <none>
I20260621 06:58:09.529003 6676 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:09.529163 6676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:09.529482 6676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:09.533633 6676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "3f079ce2cc6049fca7fd1a9818b0d13b"
format_stamp: "Formatted at 2026-06-21 06:58:09 on dist-test-slave-9zdv"
server_key: "5981ac1ecdfd173d4c122a6afc24a740"
server_key_iv: "11769d1291a03effc3aab51c1deb79ff"
server_key_version: "encryptionkey@0"
I20260621 06:58:09.534538 6676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "3f079ce2cc6049fca7fd1a9818b0d13b"
format_stamp: "Formatted at 2026-06-21 06:58:09 on dist-test-slave-9zdv"
server_key: "5981ac1ecdfd173d4c122a6afc24a740"
server_key_iv: "11769d1291a03effc3aab51c1deb79ff"
server_key_version: "encryptionkey@0"
I20260621 06:58:09.540263 6676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:58:09.543910 6691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:09.545282 6676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:09.545449 6676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "3f079ce2cc6049fca7fd1a9818b0d13b"
format_stamp: "Formatted at 2026-06-21 06:58:09 on dist-test-slave-9zdv"
server_key: "5981ac1ecdfd173d4c122a6afc24a740"
server_key_iv: "11769d1291a03effc3aab51c1deb79ff"
server_key_version: "encryptionkey@0"
I20260621 06:58:09.545646 6676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:09.572955 6676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:09.577430 6676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:09.577702 6676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:09.578749 6676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:09.580220 6676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:09.580333 6676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:09.580474 6676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:09.580541 6676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:09.612210 6676 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:46249
I20260621 06:58:09.612257 6803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:46249 every 8 connection(s)
I20260621 06:58:09.613482 6676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:09.618115 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6676
I20260621 06:58:09.618357 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:09.618968 3031 external_mini_cluster.cc:1428] Setting key 73ab8634e7d73d1766380040d60e8d6a
Jun 21 06:58:09 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:09.633553 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:09.615979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48065 (local address 127.2.245.254:44071)
0621 06:58:09.616408 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:09.616412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:09.617218 (+ 806us) server_negotiation.cc:407] Connection header received
0621 06:58:09.619101 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:09.619106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:09.619211 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:09.619339 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:09.621052 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.622052 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.623577 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.624301 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.624440 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:09.627894 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:09.627899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:09.627902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:09.627934 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:09.630446 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:09.631240 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:09.631250 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:09.631253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:09.631366 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:09.631797 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:09.631807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:09.631809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:09.632061 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:09.632213 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:09.632898 (+ 685us) server_negotiation.cc:299] Negotiation successful
0621 06:58:09.633113 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":244,"threads_started":1}
I20260621 06:58:09.634512 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:09.616391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44071 (local address 127.2.245.193:48065)
0621 06:58:09.617060 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:09.617099 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:09.618802 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:09.619593 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:09.619605 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:09.620190 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:09.620857 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:09.620884 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.622192 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.622195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:09.623424 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:09.623436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:09.624496 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:09.624501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:09.624677 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:09.625353 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:09.625375 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:09.627678 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:09.630602 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:09.630611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:09.630627 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:09.631118 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:09.631479 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:09.631483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:09.631487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:09.631669 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:09.632189 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:09.632197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:09.632535 (+ 338us) client_negotiation.cc:769] Sending connection context
0621 06:58:09.632784 (+ 249us) client_negotiation.cc:240] Negotiation successful
0621 06:58:09.633343 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":256,"threads_started":1}
I20260621 06:58:09.636035 6804 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44071
I20260621 06:58:09.636349 6804 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:09.637029 6804 heartbeater.cc:511] Master 127.2.245.254:44071 requested a full tablet report, sending...
I20260621 06:58:09.639178 6615 ts_manager.cc:194] Registered new tserver with Master: 3f079ce2cc6049fca7fd1a9818b0d13b (127.2.245.193:46249)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:09.641239 6615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:48065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:09.662487 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:09.980028 6810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:09.980333 6810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:09.980469 6810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:09.987555 6810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:09.987656 6810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:09.987766 6810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:09.997242 6810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:09.998436 6810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:09.999761 6810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:10.012770 6816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:10.012897 6815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:10.012897 6818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:10.013792 6810 server_base.cc:1034] running on GCE node
I20260621 06:58:10.014387 6810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:10.015375 6810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:10.016607 6810 hybrid_clock.cc:648] HybridClock initialized: now 1782025090016557 us; error 46 us; skew 500 ppm
I20260621 06:58:10.016924 6810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:10.020098 6810 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:10.021425 6810 webserver.cc:460] Webserver started at http://127.2.245.194:39999/ using document root <none> and password file <none>
I20260621 06:58:10.022110 6810 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:10.022239 6810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:10.022559 6810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:10.025360 6810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "ac9610ee3d0346dc886925d721c93714"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "ffee330845a3710e01df90f06a7c8991"
server_key_iv: "b0cae85b764a386f473198f409de5470"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.026108 6810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "ac9610ee3d0346dc886925d721c93714"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "ffee330845a3710e01df90f06a7c8991"
server_key_iv: "b0cae85b764a386f473198f409de5470"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.031183 6810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:58:10.034610 6825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:10.036026 6810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:10.036157 6810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "ac9610ee3d0346dc886925d721c93714"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "ffee330845a3710e01df90f06a7c8991"
server_key_iv: "b0cae85b764a386f473198f409de5470"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.036319 6810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:10.050758 6810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:10.054621 6810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:10.054855 6810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:10.055804 6810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:10.057009 6810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:10.057080 6810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:10.057173 6810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:10.057216 6810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:10.088459 6810 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:33649
I20260621 06:58:10.088495 6937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:33649 every 8 connection(s)
I20260621 06:58:10.089699 6810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:10.099105 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6810
I20260621 06:58:10.099277 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:10.099671 3031 external_mini_cluster.cc:1428] Setting key d5c419226f895b242bf5bada4056a3bb
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:10.109519 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.092116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:50645 (local address 127.2.245.254:44071)
0621 06:58:10.092281 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.092285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.093369 (+ 1084us) server_negotiation.cc:407] Connection header received
0621 06:58:10.095232 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.095238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.095329 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.095437 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.096910 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.097596 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.099149 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.099615 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.099743 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.103777 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.103782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.103785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.103822 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.106470 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.107285 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.107297 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.107300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.107421 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.107854 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.107864 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.107867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.108110 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.108258 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.108935 (+ 677us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.109154 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 06:58:10.110455 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.092487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44071 (local address 127.2.245.194:50645)
0621 06:58:10.093169 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:10.093205 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:10.094880 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:10.095675 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:10.095683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:10.096190 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:10.096741 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:10.096758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.097745 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.097749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:10.098938 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:10.098995 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.099825 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.099831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:10.100069 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.100847 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:10.100874 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:10.103556 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:10.106617 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:10.106626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:10.106643 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:10.107159 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:10.107541 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:10.107546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:10.107550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:10.107734 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:10.108236 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:10.108244 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:10.108587 (+ 343us) client_negotiation.cc:769] Sending connection context
0621 06:58:10.108848 (+ 261us) client_negotiation.cc:240] Negotiation successful
0621 06:58:10.109377 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":266,"threads_started":1}
I20260621 06:58:10.111996 6938 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44071
I20260621 06:58:10.112283 6938 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:10.112979 6938 heartbeater.cc:511] Master 127.2.245.254:44071 requested a full tablet report, sending...
I20260621 06:58:10.114483 6615 ts_manager.cc:194] Registered new tserver with Master: ac9610ee3d0346dc886925d721c93714 (127.2.245.194:33649)
I20260621 06:58:10.115363 6615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:50645
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:10.141844 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:10.458359 6943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:10.458671 6943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:10.458842 6943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:10.465822 6943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:10.465965 6943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:10.466184 6943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:10.475641 6943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:10.476909 6943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:10.478196 6943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:10.491149 6951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:10.491127 6949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:10.491586 6948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:10.491725 6943 server_base.cc:1034] running on GCE node
I20260621 06:58:10.492353 6943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:10.493212 6943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:10.494489 6943 hybrid_clock.cc:648] HybridClock initialized: now 1782025090494430 us; error 52 us; skew 500 ppm
I20260621 06:58:10.494802 6943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:10.497864 6943 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:10.499341 6943 webserver.cc:460] Webserver started at http://127.2.245.195:37763/ using document root <none> and password file <none>
I20260621 06:58:10.499977 6943 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:10.500114 6943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:10.500418 6943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:10.503213 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "e8c846e1559a4a6b9632ae7d351a646b"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "b78141ae3b0745291b76c93f39b5d6f2"
server_key_iv: "926f75bb3590db289ce170387a576c4f"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.503974 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "e8c846e1559a4a6b9632ae7d351a646b"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "b78141ae3b0745291b76c93f39b5d6f2"
server_key_iv: "926f75bb3590db289ce170387a576c4f"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.509124 6943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:10.512472 6958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:10.513772 6943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 06:58:10.513939 6943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "e8c846e1559a4a6b9632ae7d351a646b"
format_stamp: "Formatted at 2026-06-21 06:58:10 on dist-test-slave-9zdv"
server_key: "b78141ae3b0745291b76c93f39b5d6f2"
server_key_iv: "926f75bb3590db289ce170387a576c4f"
server_key_version: "encryptionkey@0"
I20260621 06:58:10.514143 6943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:10.526245 6943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:10.530308 6943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:10.530599 6943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:10.531608 6943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:10.532924 6943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:10.533030 6943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:10.533192 6943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:10.533378 6943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260621 06:58:10.566365 6943 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:33401
I20260621 06:58:10.566398 7070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:33401 every 8 connection(s)
I20260621 06:58:10.567687 6943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:10.569530 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 6943
I20260621 06:58:10.569732 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:10.570228 3031 external_mini_cluster.cc:1428] Setting key 9dab6b84112d6f03315ce315139ffcd8
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:10.587044 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.570234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:44711 (local address 127.2.245.254:44071)
0621 06:58:10.570433 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.570441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.571769 (+ 1328us) server_negotiation.cc:407] Connection header received
0621 06:58:10.573792 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.573797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.573885 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.573996 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.575550 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.576303 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.577793 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.578179 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.578318 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.581569 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.581574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.581577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.581618 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.584111 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.584799 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.584805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.584810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.584940 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.585347 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.585352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.585355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.585645 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.585794 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.586464 (+ 670us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.586669 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 06:58:10.587939 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.570779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44071 (local address 127.2.245.195:44711)
0621 06:58:10.571623 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:10.571671 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:10.573534 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:10.574231 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:10.574239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:10.574732 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:10.575384 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:10.575401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.576448 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.576453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:10.577656 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:10.577667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.578394 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.578403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:10.578642 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.579321 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:10.579342 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:10.581349 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:10.584248 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:10.584256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:10.584271 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:10.584676 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:10.585049 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:10.585054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:10.585057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:10.585227 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:10.585770 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:10.585778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:10.586111 (+ 333us) client_negotiation.cc:769] Sending connection context
0621 06:58:10.586363 (+ 252us) client_negotiation.cc:240] Negotiation successful
0621 06:58:10.586870 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":329,"threads_started":1}
I20260621 06:58:10.589231 7071 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44071
I20260621 06:58:10.589571 7071 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:10.590337 7071 heartbeater.cc:511] Master 127.2.245.254:44071 requested a full tablet report, sending...
I20260621 06:58:10.591897 6615 ts_manager.cc:194] Registered new tserver with Master: e8c846e1559a4a6b9632ae7d351a646b (127.2.245.195:33401)
I20260621 06:58:10.592700 6615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:44711
I20260621 06:58:10.597822 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:10.644338 6804 heartbeater.cc:503] Master 127.2.245.254:44071 was elected leader, sending a full tablet report...
I20260621 06:58:10.646842 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.631609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.2.245.193:46249)
0621 06:58:10.632245 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.632254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.632285 (+ 31us) server_negotiation.cc:407] Connection header received
0621 06:58:10.632483 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.632490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.632704 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.632879 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.634301 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.635354 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.636574 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.637131 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.637292 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.641036 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.641046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.641060 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.641095 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.643869 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.644659 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.644674 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.644677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.644826 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.645217 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.645223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.645226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.645559 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.645898 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.646223 (+ 325us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.646499 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":304,"threads_started":1}
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:10.664166 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.651033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.2.245.254:44071)
0621 06:58:10.651223 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.651227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.651264 (+ 37us) server_negotiation.cc:407] Connection header received
0621 06:58:10.651445 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.651449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.651506 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.651580 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.652745 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.653487 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.654660 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.655094 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.655200 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.659112 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.659117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.659121 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.659153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.661574 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.662230 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.662237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.662240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.662382 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.662749 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.662759 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.662761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.663106 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.663358 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.663664 (+ 306us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.663867 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260621 06:58:10.664742 6615 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60798
I20260621 06:58:10.665663 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6676
I20260621 06:58:10.677865 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.666624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.2.245.193:46249)
0621 06:58:10.666832 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.666839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.667028 (+ 189us) server_negotiation.cc:407] Connection header received
0621 06:58:10.667218 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.667230 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.667318 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.667418 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.668543 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.669206 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.670289 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.670680 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.670811 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.672891 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.672901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.672904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.672934 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.675366 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.675990 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.675996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.676000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.676125 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.676491 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.676497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.676501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.676749 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.676992 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.677310 (+ 318us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.677513 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 06:58:10.678478 6799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:10.758543 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6676
I20260621 06:58:10.774822 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6810
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:58:10.790210 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.775883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.2.245.194:33649)
0621 06:58:10.776473 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.776480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.776505 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:58:10.776579 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.776585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.776796 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.776940 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.778040 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.779271 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.780440 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.780987 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.781075 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.785003 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.785013 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.785026 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.785056 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.787932 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.788514 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.788520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.788523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.788645 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.788963 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.788967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.788968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.789280 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.789513 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.789722 (+ 209us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.789925 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":312,"threads_started":1}
I20260621 06:58:10.791559 6933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:10.872303 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6810
I20260621 06:58:10.887878 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6943
Jun 21 06:58:10 dist-test-slave-9zdv krb5kdc[6570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:58:10.903556 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:10.888808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54306 (local address 127.2.245.195:33401)
0621 06:58:10.889436 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:10.889443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:10.889472 (+ 29us) server_negotiation.cc:407] Connection header received
0621 06:58:10.889530 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:10.889536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:10.889738 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:10.889918 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:10.891146 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.892138 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.893380 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:10.893944 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:10.894088 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:10.898215 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:10.898225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:10.898240 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:10.898270 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:10.900975 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.901586 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.901592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.901594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.901700 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:10.902044 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:10.902049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:10.902052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:10.902357 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:10.902653 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:10.902929 (+ 276us) server_negotiation.cc:299] Negotiation successful
0621 06:58:10.903219 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":338,"threads_started":1}
I20260621 06:58:10.904812 7066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:10.989142 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6943
I20260621 06:58:11.004576 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 6583
I20260621 06:58:11.005666 6645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:11.105121 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 6583
2026-06-21T06:58:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3787 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:11 dist-test-slave-9zdv krb5kdc[7090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:11 dist-test-slave-9zdv krb5kdc[7090](info): set up 2 sockets
Jun 21 06:58:11 dist-test-slave-9zdv krb5kdc[7090](info): commencing operation
krb5kdc: starting...
W20260621 06:58:12.259517 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:12 dist-test-slave-9zdv krb5kdc[7090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:12.454694 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38409
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:12.771064 7103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:12.771339 7103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:12.771512 7103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:12.778474 7103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:12.778601 7103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:12.778676 7103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:12.778769 7103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:12.778828 7103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:12.788292 7103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:12.789584 7103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:12.790815 7103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:12.801751 7108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:12.801823 7111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:12.801795 7109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:12.802975 7103 server_base.cc:1034] running on GCE node
I20260621 06:58:12.803722 7103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:12.804998 7103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:12.806228 7103 hybrid_clock.cc:648] HybridClock initialized: now 1782025092806199 us; error 48 us; skew 500 ppm
I20260621 06:58:12.806613 7103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:12 dist-test-slave-9zdv krb5kdc[7090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:12.809758 7103 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:12.810883 7103 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:12.811048 7103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:12.811458 7103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:12.814440 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "7d1de5bef18642fab3d83dde87b6db5d"
format_stamp: "Formatted at 2026-06-21 06:58:12 on dist-test-slave-9zdv"
server_key: "980b3cbfe59c5d4afb19c9ab4d569b55"
server_key_iv: "203e02e3450eec577e7c2e87dfb063f2"
server_key_version: "encryptionkey@0"
I20260621 06:58:12.815536 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "7d1de5bef18642fab3d83dde87b6db5d"
format_stamp: "Formatted at 2026-06-21 06:58:12 on dist-test-slave-9zdv"
server_key: "980b3cbfe59c5d4afb19c9ab4d569b55"
server_key_iv: "203e02e3450eec577e7c2e87dfb063f2"
server_key_version: "encryptionkey@0"
I20260621 06:58:12.820358 7103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260621 06:58:12.823750 7117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:12.825022 7103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260621 06:58:12.825199 7103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "7d1de5bef18642fab3d83dde87b6db5d"
format_stamp: "Formatted at 2026-06-21 06:58:12 on dist-test-slave-9zdv"
server_key: "980b3cbfe59c5d4afb19c9ab4d569b55"
server_key_iv: "203e02e3450eec577e7c2e87dfb063f2"
server_key_version: "encryptionkey@0"
I20260621 06:58:12.825412 7103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:12.842878 7103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:12.847153 7103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:12.847420 7103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:12.865685 7103 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:38409
I20260621 06:58:12.865703 7168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:38409 every 8 connection(s)
I20260621 06:58:12.866593 7103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:12.870183 7169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:12.872761 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7103
I20260621 06:58:12.873001 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:12.873790 3031 external_mini_cluster.cc:1428] Setting key b2211695cfb67760d133e381677cb17f
I20260621 06:58:12.881837 7169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: Bootstrap starting.
I20260621 06:58:12.884815 7169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:12.885753 7169 log.cc:824] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: Log is configured to *not* fsync() on all Append() calls
Jun 21 06:58:12 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:12.888623 7169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: No bootstrap required, opened a new log
I20260621 06:58:12.894850 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:12.875918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.2.245.254:38409)
0621 06:58:12.876684 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:12.876697 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:12.876736 (+ 39us) server_negotiation.cc:407] Connection header received
0621 06:58:12.878307 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:12.878328 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:12.878691 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:12.879100 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:12.880541 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:12.881802 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:12.883037 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:12.883708 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:12.883911 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:12.888038 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:12.888045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:12.888065 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:12.888101 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:12.891024 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:12.891763 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:12.891768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:12.891774 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:12.891913 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:12.892278 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:12.892281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:12.892282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:12.892794 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:12.892995 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:12.893430 (+ 435us) server_negotiation.cc:299] Negotiation successful
0621 06:58:12.893830 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":342,"threads_started":1}
I20260621 06:58:12.896873 7169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } }
I20260621 06:58:12.897220 7169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:12.897387 7169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1de5bef18642fab3d83dde87b6db5d, State: Initialized, Role: FOLLOWER
I20260621 06:58:12.897964 7169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } }
I20260621 06:58:12.898159 7169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:12.898250 7169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:12.898366 7169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:12.900451 7169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } }
I20260621 06:58:12.900892 7169 leader_election.cc:302] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d1de5bef18642fab3d83dde87b6db5d; no voters:
I20260621 06:58:12.901391 7169 leader_election.cc:288] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:12.901530 7174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:12.902256 7174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [term 1 LEADER]: Becoming Leader. State: Replica: 7d1de5bef18642fab3d83dde87b6db5d, State: Running, Role: LEADER
I20260621 06:58:12.902786 7174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } }
I20260621 06:58:12.903538 7169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:12.905882 7175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d1de5bef18642fab3d83dde87b6db5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } } }
I20260621 06:58:12.905884 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d1de5bef18642fab3d83dde87b6db5d. Latest consensus state: current_term: 1 leader_uuid: "7d1de5bef18642fab3d83dde87b6db5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1de5bef18642fab3d83dde87b6db5d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38409 } } }
I20260621 06:58:12.906311 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:12.906311 7175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:12.912156 7180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:12.918503 7180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:12.928198 7180 catalog_manager.cc:1347] Generated new cluster ID: 27ce4404499c49c3890c9d3ca396b0d6
I20260621 06:58:12.928332 7180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:12.947744 7180 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:12.948977 7180 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:12.959571 7180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: Generated new TSK 0
I20260621 06:58:12.960378 7180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:13.024598 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:13.346428 7195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:13.346738 7195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:13.346838 7195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:13.354398 7195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:13.354503 7195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:13.354625 7195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:13.364313 7195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:13.365541 7195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:13.366791 7195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:13.379705 7201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:13.379631 7200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:13.379739 7203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:13.380476 7195 server_base.cc:1034] running on GCE node
I20260621 06:58:13.381052 7195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:13.382048 7195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:13.383260 7195 hybrid_clock.cc:648] HybridClock initialized: now 1782025093383228 us; error 55 us; skew 500 ppm
I20260621 06:58:13.383572 7195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:13 dist-test-slave-9zdv krb5kdc[7090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025093, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:13.386756 7195 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:13.388006 7195 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:13.388099 7195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:13.388412 7195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:13.391371 7195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "1b6c6b7e099c4c66873a5fc02f413f06"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "be923c36e75a51318dcd485b979ab263"
server_key_iv: "089384f72dfb79c82fcc3947d08dbf1e"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.392233 7195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "1b6c6b7e099c4c66873a5fc02f413f06"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "be923c36e75a51318dcd485b979ab263"
server_key_iv: "089384f72dfb79c82fcc3947d08dbf1e"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.397526 7195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:13.401075 7209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.402484 7195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:13.402669 7195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "1b6c6b7e099c4c66873a5fc02f413f06"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "be923c36e75a51318dcd485b979ab263"
server_key_iv: "089384f72dfb79c82fcc3947d08dbf1e"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.402894 7195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:13.425261 7195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:13.429525 7195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:13.429821 7195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:13.430756 7195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:13.432107 7195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:13.432191 7195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.432348 7195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:13.432442 7195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.465672 7195 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:38859
I20260621 06:58:13.465728 7321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:38859 every 8 connection(s)
I20260621 06:58:13.466572 7195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:13.473620 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7195
I20260621 06:58:13.473842 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:13.474638 3031 external_mini_cluster.cc:1428] Setting key 94b8161ccd707b1ba7e76271bdb09849
Jun 21 06:58:13 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025093, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:13.489332 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:13.470177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48817 (local address 127.2.245.254:38409)
0621 06:58:13.470640 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:13.470647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:13.470890 (+ 243us) server_negotiation.cc:407] Connection header received
0621 06:58:13.472751 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:13.472756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:13.472842 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:13.472983 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:13.475132 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.476148 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.478453 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.478848 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.479008 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:13.483567 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:13.483572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:13.483575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:13.483609 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:13.486190 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:13.487000 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:13.487006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:13.487008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:13.487140 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:13.487581 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:13.487593 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:13.487595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:13.487880 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:13.488041 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:13.488740 (+ 699us) server_negotiation.cc:299] Negotiation successful
0621 06:58:13.488958 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":236,"threads_started":1}
I20260621 06:58:13.490425 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:13.469832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38409 (local address 127.2.245.193:48817)
0621 06:58:13.470723 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:13.470779 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:13.472466 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:13.473228 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:13.473240 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:13.474058 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:13.474907 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:13.474932 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.476339 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.476343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:13.478272 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:13.478291 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.479132 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.479142 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:13.479466 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:13.480217 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:13.480253 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:13.483321 (+ 3068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:13.486350 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:13.486360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:13.486379 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:13.486828 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:13.487260 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:13.487266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:13.487269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:13.487457 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:13.488020 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:13.488028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:13.488370 (+ 342us) client_negotiation.cc:769] Sending connection context
0621 06:58:13.488637 (+ 267us) client_negotiation.cc:240] Negotiation successful
0621 06:58:13.489184 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":329,"threads_started":1}
I20260621 06:58:13.492244 7322 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38409
I20260621 06:58:13.492549 7322 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:13.493175 7322 heartbeater.cc:511] Master 127.2.245.254:38409 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:13.495523 7134 ts_manager.cc:194] Registered new tserver with Master: 1b6c6b7e099c4c66873a5fc02f413f06 (127.2.245.193:38859)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:13.497001 7134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:48817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:13.520164 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:13.840157 7328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:13.840458 7328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:13.840623 7328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:13.848040 7328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:13.848182 7328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:13.848364 7328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:13.858023 7328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:13.859372 7328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:13.860754 7328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:13.876287 7333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:13.876223 7336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:13.876259 7334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:13.877427 7328 server_base.cc:1034] running on GCE node
I20260621 06:58:13.878034 7328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:13.878994 7328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:13.880263 7328 hybrid_clock.cc:648] HybridClock initialized: now 1782025093880206 us; error 51 us; skew 500 ppm
I20260621 06:58:13.880573 7328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:13 dist-test-slave-9zdv krb5kdc[7090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025093, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:13.887960 7328 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:13.889135 7328 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:13.889267 7328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:13.889567 7328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:13.892426 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "6d4f3b3cea8e4c6183802add2f6aee70"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "028b7aa46ff57c0c2243e1cc8aa6ea3c"
server_key_iv: "73cbeee4e24e5ca11b4789863dc1bafb"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.893226 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "6d4f3b3cea8e4c6183802add2f6aee70"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "028b7aa46ff57c0c2243e1cc8aa6ea3c"
server_key_iv: "73cbeee4e24e5ca11b4789863dc1bafb"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.898240 7328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260621 06:58:13.901922 7342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.903273 7328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 06:58:13.903439 7328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "6d4f3b3cea8e4c6183802add2f6aee70"
format_stamp: "Formatted at 2026-06-21 06:58:13 on dist-test-slave-9zdv"
server_key: "028b7aa46ff57c0c2243e1cc8aa6ea3c"
server_key_iv: "73cbeee4e24e5ca11b4789863dc1bafb"
server_key_version: "encryptionkey@0"
I20260621 06:58:13.903692 7328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:13.916447 7328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:13.920447 7328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:13.920759 7328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:13.921727 7328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:13.923069 7328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:13.923178 7328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.923302 7328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:13.923391 7328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:13.956447 7328 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38791
I20260621 06:58:13.956540 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38791 every 8 connection(s)
I20260621 06:58:13.957345 7328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:13.958724 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7328
I20260621 06:58:13.958922 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:13.959436 3031 external_mini_cluster.cc:1428] Setting key 28a1508e45df56260869cbe6a08cc016
Jun 21 06:58:13 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025093, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:58:13.979132 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:13.960067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:49987 (local address 127.2.245.254:38409)
0621 06:58:13.960235 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:13.960240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:13.961893 (+ 1653us) server_negotiation.cc:407] Connection header received
0621 06:58:13.964676 (+ 2783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:13.964680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:13.964764 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:13.964843 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:13.966991 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.967701 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.969348 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.969770 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.969906 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:13.973304 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:13.973309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:13.973312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:13.973343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:13.976078 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:13.976830 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:13.976838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:13.976843 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:13.976980 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:13.977393 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:13.977400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:13.977403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:13.977661 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:13.977827 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:13.978524 (+ 697us) server_negotiation.cc:299] Negotiation successful
0621 06:58:13.978761 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 06:58:13.980268 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:13.960423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38409 (local address 127.2.245.194:49987)
0621 06:58:13.961713 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:13.961767 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:13.964354 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:13.965138 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:13.965151 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:13.965932 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:13.966743 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:13.966770 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.967887 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.967892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:13.969198 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:13.969210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:13.970013 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:13.970020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:13.970236 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:13.970918 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:13.970995 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:13.973068 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:13.976229 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:13.976238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:13.976255 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:13.976709 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:13.977094 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:13.977098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:13.977102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:13.977278 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:13.977790 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:13.977809 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:13.978133 (+ 324us) client_negotiation.cc:769] Sending connection context
0621 06:58:13.978455 (+ 322us) client_negotiation.cc:240] Negotiation successful
0621 06:58:13.979002 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":837,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
I20260621 06:58:13.981810 7455 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38409
I20260621 06:58:13.982115 7455 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:13.982892 7455 heartbeater.cc:511] Master 127.2.245.254:38409 requested a full tablet report, sending...
I20260621 06:58:13.984457 7134 ts_manager.cc:194] Registered new tserver with Master: 6d4f3b3cea8e4c6183802add2f6aee70 (127.2.245.194:38791)
I20260621 06:58:13.985270 7134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:49987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:14.003304 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:14.314534 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:14.314801 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:14.314976 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:14.322058 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:14.322161 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:14.322376 7460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:14.332170 7460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:14.333449 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:14.334847 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:14.347144 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:14.347182 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:14.347354 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:14.348199 7460 server_base.cc:1034] running on GCE node
I20260621 06:58:14.348786 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:14.349718 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:14.350980 7460 hybrid_clock.cc:648] HybridClock initialized: now 1782025094350902 us; error 51 us; skew 500 ppm
I20260621 06:58:14.351285 7460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:14 dist-test-slave-9zdv krb5kdc[7090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025094, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:14.354466 7460 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:14.355634 7460 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:14.355733 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:14.355978 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:14.358752 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7"
format_stamp: "Formatted at 2026-06-21 06:58:14 on dist-test-slave-9zdv"
server_key: "17286dd9185e5c52c4def362c35632ef"
server_key_iv: "c0cfe0dac191806d317057a7636234cc"
server_key_version: "encryptionkey@0"
I20260621 06:58:14.359601 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7"
format_stamp: "Formatted at 2026-06-21 06:58:14 on dist-test-slave-9zdv"
server_key: "17286dd9185e5c52c4def362c35632ef"
server_key_iv: "c0cfe0dac191806d317057a7636234cc"
server_key_version: "encryptionkey@0"
I20260621 06:58:14.364425 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:58:14.367725 7474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:14.368989 7460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:14.369108 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7"
format_stamp: "Formatted at 2026-06-21 06:58:14 on dist-test-slave-9zdv"
server_key: "17286dd9185e5c52c4def362c35632ef"
server_key_iv: "c0cfe0dac191806d317057a7636234cc"
server_key_version: "encryptionkey@0"
I20260621 06:58:14.369287 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:14.389905 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:14.394004 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:14.394277 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:14.395252 7460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:14.396520 7460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:14.396649 7460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:14.396818 7460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:14.396908 7460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:14.428397 7460 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:35377
I20260621 06:58:14.428450 7586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:35377 every 8 connection(s)
I20260621 06:58:14.429337 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:14.431666 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7460
I20260621 06:58:14.431867 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:14.432379 3031 external_mini_cluster.cc:1428] Setting key 3d0247f332747678eef4d948e97c18c5
Jun 21 06:58:14 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025094, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:14.449136 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.433266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:47051 (local address 127.2.245.254:38409)
0621 06:58:14.433468 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.433475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.433745 (+ 270us) server_negotiation.cc:407] Connection header received
0621 06:58:14.436074 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.436078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.436174 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.436259 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:14.437661 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.438372 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.439910 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.440303 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.440398 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.443599 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:14.443604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:14.443607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:14.443636 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:14.446044 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:14.446930 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:14.446952 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:14.446956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:14.447107 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:14.447579 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:14.447582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:14.447584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:14.447961 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:14.448099 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.448732 (+ 633us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.448885 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 06:58:14.450078 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.432662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38409 (local address 127.2.245.195:47051)
0621 06:58:14.433548 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.433609 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.435843 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.436453 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.436461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.436974 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:14.437513 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.437530 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.438490 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.438493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.439765 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.439776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.440481 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.440485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.440662 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.441343 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:14.441363 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:14.443419 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:14.446186 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:14.446193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:14.446207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:14.446797 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:14.447228 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:14.447234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:14.447240 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:14.447481 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:14.448095 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:14.448106 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:14.448397 (+ 291us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.448613 (+ 216us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.448978 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":343,"threads_started":1}
I20260621 06:58:14.451423 7587 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38409
I20260621 06:58:14.451702 7587 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:14.452329 7587 heartbeater.cc:511] Master 127.2.245.254:38409 requested a full tablet report, sending...
I20260621 06:58:14.453670 7134 ts_manager.cc:194] Registered new tserver with Master: a3dbf3b64af84a8eae9d0b2deca9d4b7 (127.2.245.195:35377)
I20260621 06:58:14.454452 7134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:47051
I20260621 06:58:14.459914 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:14.476570 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.464178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36806 (local address 127.2.245.254:38409)
0621 06:58:14.464436 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.464443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.464808 (+ 365us) server_negotiation.cc:407] Connection header received
0621 06:58:14.465001 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.465006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.465068 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.465173 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:14.466586 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.467364 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.468807 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.469242 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.469368 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.471504 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:14.471508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:14.471511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:14.471540 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:14.473959 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:14.474584 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:14.474590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:14.474594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:14.474730 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:14.475170 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:14.475177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:14.475182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:14.475494 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:14.475772 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.476067 (+ 295us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.476269 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:58:14.480217 7134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:14.486794 7134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:14.500656 7322 heartbeater.cc:503] Master 127.2.245.254:38409 was elected leader, sending a full tablet report...
I20260621 06:58:14.508131 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.498140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:35377 (local address 127.0.0.1:34568)
0621 06:58:14.499045 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.499089 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.499283 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.500559 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.500566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.500593 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:14.500988 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.501002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.502538 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.502543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.505055 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.505066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.507118 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.507128 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.507408 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.507464 (+ 56us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.507580 (+ 116us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.507771 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":5364,"threads_started":1}
I20260621 06:58:14.508761 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.498434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.2.245.195:35377)
0621 06:58:14.499901 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.499909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.499939 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:58:14.500037 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.500047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.500313 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.501153 (+ 840us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:14.501381 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.502389 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.505302 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.506860 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.507930 (+ 1070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.508162 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.508234 (+ 72us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.508410 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":1132,"threads_started":1}
I20260621 06:58:14.512570 7522 tablet_service.cc:1467] Processing CreateTablet for tablet a18496dfdc144a70b108b845fceb950d (DEFAULT_TABLE table=test-table [id=dde9f9b080d547b3bb19276c0f6e6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:14.514012 7522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18496dfdc144a70b108b845fceb950d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:14.514322 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.498260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42906)
0621 06:58:14.503772 (+ 5512us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.503797 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.503909 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.505026 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.505031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.505058 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:14.505559 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.505570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.508885 (+ 3315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.508892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.511700 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.511715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.513488 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.513495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.513686 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.513742 (+ 56us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.513859 (+ 117us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.513994 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5378,"thread_start_us":695,"threads_started":1}
I20260621 06:58:14.514392 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.499035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.2.245.193:38859)
0621 06:58:14.504468 (+ 5433us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.504477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.504506 (+ 29us) server_negotiation.cc:407] Connection header received
0621 06:58:14.504587 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.504595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.504815 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.505764 (+ 949us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:14.507079 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.508683 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.511868 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.513265 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.513404 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.513670 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.513915 (+ 245us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.514090 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5261,"thread_start_us":5104,"threads_started":1}
I20260621 06:58:14.517524 7257 tablet_service.cc:1467] Processing CreateTablet for tablet a18496dfdc144a70b108b845fceb950d (DEFAULT_TABLE table=test-table [id=dde9f9b080d547b3bb19276c0f6e6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:14.518765 7257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18496dfdc144a70b108b845fceb950d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:14.519094 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.506451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38791 (local address 127.0.0.1:57184)
0621 06:58:14.507532 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.507551 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.507732 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.508561 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.508565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.508594 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:14.509710 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.509721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.511392 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.511399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.513364 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.513377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.518269 (+ 4892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.518275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.518469 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.518500 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.518600 (+ 100us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.518731 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":809,"threads_started":1}
I20260621 06:58:14.519248 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.504331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.2.245.194:38791)
0621 06:58:14.507045 (+ 2714us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.507054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.507653 (+ 599us) server_negotiation.cc:407] Connection header received
0621 06:58:14.507901 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.507909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.508125 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.508310 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:14.509874 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.511263 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.513541 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.515054 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.515194 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.515367 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.518678 (+ 3311us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.518862 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2519,"thread_start_us":2383,"threads_started":1}
I20260621 06:58:14.522543 7390 tablet_service.cc:1467] Processing CreateTablet for tablet a18496dfdc144a70b108b845fceb950d (DEFAULT_TABLE table=test-table [id=dde9f9b080d547b3bb19276c0f6e6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:14.523864 7390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18496dfdc144a70b108b845fceb950d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:14.525723 7605 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Bootstrap starting.
I20260621 06:58:14.529018 7605 tablet_bootstrap.cc:654] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:14.530138 7605 log.cc:824] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:14.535914 7607 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Bootstrap starting.
I20260621 06:58:14.537916 7605 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: No bootstrap required, opened a new log
I20260621 06:58:14.538218 7605 ts_tablet_manager.cc:1397] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260621 06:58:14.538872 7608 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Bootstrap starting.
I20260621 06:58:14.539644 7607 tablet_bootstrap.cc:654] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:14.540838 7607 log.cc:824] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:14.541870 7608 tablet_bootstrap.cc:654] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:14.542735 7608 log.cc:824] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:14.545311 7607 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: No bootstrap required, opened a new log
I20260621 06:58:14.545392 7608 tablet_bootstrap.cc:492] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: No bootstrap required, opened a new log
I20260621 06:58:14.545585 7607 ts_tablet_manager.cc:1397] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260621 06:58:14.545595 7608 ts_tablet_manager.cc:1397] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260621 06:58:14.547447 7605 raft_consensus.cc:348] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.547802 7605 raft_consensus.cc:374] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:14.547920 7605 raft_consensus.cc:729] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3dbf3b64af84a8eae9d0b2deca9d4b7, State: Initialized, Role: FOLLOWER
I20260621 06:58:14.548583 7605 consensus_queue.cc:260] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.549657 7587 heartbeater.cc:503] Master 127.2.245.254:38409 was elected leader, sending a full tablet report...
I20260621 06:58:14.550295 7605 ts_tablet_manager.cc:1428] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260621 06:58:14.555747 7608 raft_consensus.cc:348] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.556214 7608 raft_consensus.cc:374] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:14.556330 7608 raft_consensus.cc:729] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4f3b3cea8e4c6183802add2f6aee70, State: Initialized, Role: FOLLOWER
I20260621 06:58:14.556471 7607 raft_consensus.cc:348] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.556855 7607 raft_consensus.cc:374] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:14.556967 7607 raft_consensus.cc:729] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6b7e099c4c66873a5fc02f413f06, State: Initialized, Role: FOLLOWER
I20260621 06:58:14.557144 7608 consensus_queue.cc:260] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.557497 7607 consensus_queue.cc:260] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.558302 7455 heartbeater.cc:503] Master 127.2.245.254:38409 was elected leader, sending a full tablet report...
I20260621 06:58:14.559000 7608 ts_tablet_manager.cc:1428] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260621 06:58:14.559188 7607 ts_tablet_manager.cc:1428] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260621 06:58:14.680713 7588 tablet.cc:2369] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:14.709306 7456 tablet.cc:2369] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:14.718820 7323 tablet.cc:2369] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:58:14.872800 7613 raft_consensus.cc:482] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:14.873139 7613 raft_consensus.cc:504] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.875223 7613 leader_election.cc:288] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3dbf3b64af84a8eae9d0b2deca9d4b7 (127.2.245.195:35377), 6d4f3b3cea8e4c6183802add2f6aee70 (127.2.245.194:38791)
I20260621 06:58:14.882642 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.876383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:49139 (local address 127.2.245.194:38791)
0621 06:58:14.876571 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.876578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.877018 (+ 440us) server_negotiation.cc:407] Connection header received
0621 06:58:14.877264 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.877271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.877359 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.877461 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:14.878038 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.878902 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.880622 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.881770 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.881926 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.882005 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.882323 (+ 318us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.882413 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 06:58:14.882850 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.876202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38791 (local address 127.2.245.193:49139)
0621 06:58:14.876885 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.876902 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.877096 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.877485 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.877490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.877515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:14.877857 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.877866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.879088 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.879092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.880463 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.880475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.881962 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.881967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.882141 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.882179 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.882290 (+ 111us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.882424 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"spinlock_wait_cycles":23424,"thread_start_us":358,"threads_started":1}
I20260621 06:58:14.883064 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.875488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:39825 (local address 127.2.245.195:35377)
0621 06:58:14.875701 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.875708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.876182 (+ 474us) server_negotiation.cc:407] Connection header received
0621 06:58:14.876517 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.876527 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.876621 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.876684 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:14.877529 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.878439 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.880840 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.882013 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.882165 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.882212 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.882626 (+ 414us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.882791 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 06:58:14.883144 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.875432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:35377 (local address 127.2.245.193:39825)
0621 06:58:14.876086 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:14.876107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:14.876304 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:14.876913 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:14.876916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:14.876937 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:14.877368 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.877382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.878579 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.878584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.880701 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:14.880713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.882277 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.882284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:14.882448 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.882475 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 06:58:14.882569 (+ 94us) client_negotiation.cc:240] Negotiation successful
0621 06:58:14.882683 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":2,"thread_start_us":357,"threads_started":1}
I20260621 06:58:14.884106 7542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18496dfdc144a70b108b845fceb950d" candidate_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" is_pre_election: true
I20260621 06:58:14.884106 7410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18496dfdc144a70b108b845fceb950d" candidate_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" is_pre_election: true
I20260621 06:58:14.884635 7542 raft_consensus.cc:2454] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6c6b7e099c4c66873a5fc02f413f06 in term 0.
I20260621 06:58:14.884635 7410 raft_consensus.cc:2454] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6c6b7e099c4c66873a5fc02f413f06 in term 0.
I20260621 06:58:14.885274 7210 leader_election.cc:302] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6b7e099c4c66873a5fc02f413f06, 6d4f3b3cea8e4c6183802add2f6aee70; no voters:
I20260621 06:58:14.885704 7613 raft_consensus.cc:2781] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:14.885866 7613 raft_consensus.cc:482] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:14.886003 7613 raft_consensus.cc:3037] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:14.888224 7613 raft_consensus.cc:504] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.888825 7613 leader_election.cc:288] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [CANDIDATE]: Term 1 election: Requested vote from peers a3dbf3b64af84a8eae9d0b2deca9d4b7 (127.2.245.195:35377), 6d4f3b3cea8e4c6183802add2f6aee70 (127.2.245.194:38791)
I20260621 06:58:14.889240 7542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18496dfdc144a70b108b845fceb950d" candidate_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7"
I20260621 06:58:14.889302 7410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18496dfdc144a70b108b845fceb950d" candidate_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4f3b3cea8e4c6183802add2f6aee70"
I20260621 06:58:14.889460 7542 raft_consensus.cc:3037] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:14.889536 7410 raft_consensus.cc:3037] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:14.891788 7542 raft_consensus.cc:2454] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6c6b7e099c4c66873a5fc02f413f06 in term 1.
I20260621 06:58:14.891835 7410 raft_consensus.cc:2454] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6c6b7e099c4c66873a5fc02f413f06 in term 1.
I20260621 06:58:14.892220 7210 leader_election.cc:302] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6b7e099c4c66873a5fc02f413f06, 6d4f3b3cea8e4c6183802add2f6aee70; no voters:
I20260621 06:58:14.892524 7613 raft_consensus.cc:2781] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:14.892938 7613 raft_consensus.cc:686] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 1 LEADER]: Becoming Leader. State: Replica: 1b6c6b7e099c4c66873a5fc02f413f06, State: Running, Role: LEADER
I20260621 06:58:14.893409 7613 consensus_queue.cc:237] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } }
I20260621 06:58:14.898343 7133 catalog_manager.cc:5495] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6c6b7e099c4c66873a5fc02f413f06 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6c6b7e099c4c66873a5fc02f413f06" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 } health_report { overall_health: UNKNOWN } } }
I20260621 06:58:14.963027 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:14.957073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.2.245.193:38859)
0621 06:58:14.957298 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:14.957305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:14.957385 (+ 80us) server_negotiation.cc:407] Connection header received
0621 06:58:14.957632 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:14.957636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:14.957716 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:14.957875 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:14.958262 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.959036 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.960466 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:14.961126 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:14.961305 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:14.961782 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:14.962409 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:14.962576 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:14.962624 (+ 48us) server_negotiation.cc:299] Negotiation successful
0621 06:58:14.962736 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260621 06:58:14.970870 7410 raft_consensus.cc:1261] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 1 FOLLOWER]: Refusing update from remote peer 1b6c6b7e099c4c66873a5fc02f413f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:14.971066 7542 raft_consensus.cc:1261] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 1 FOLLOWER]: Refusing update from remote peer 1b6c6b7e099c4c66873a5fc02f413f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:14.971935 7613 consensus_queue.cc:1035] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d4f3b3cea8e4c6183802add2f6aee70" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:14.972270 7616 consensus_queue.cc:1035] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3dbf3b64af84a8eae9d0b2deca9d4b7" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:14.983839 7619 mvcc.cc:205] Tried to move back new op lower bound from 7299174788991795200 to 7299174788687360000. Current Snapshot: MvccSnapshot[applied={T|T < 7299174788991795200}]
I20260621 06:58:14.997648 7133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:14.997957 7133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:15.001636 7133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d1de5bef18642fab3d83dde87b6db5d: Sending DeleteTablet for 3 replicas of tablet a18496dfdc144a70b108b845fceb950d
I20260621 06:58:15.002846 7522 tablet_service.cc:1514] Processing DeleteTablet for tablet a18496dfdc144a70b108b845fceb950d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:14 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:34568
I20260621 06:58:15.002830 7257 tablet_service.cc:1514] Processing DeleteTablet for tablet a18496dfdc144a70b108b845fceb950d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:14 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:42906
I20260621 06:58:15.002895 7390 tablet_service.cc:1514] Processing DeleteTablet for tablet a18496dfdc144a70b108b845fceb950d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:14 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:57184
I20260621 06:58:15.005432 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7195
I20260621 06:58:15.035617 7630 tablet_replica.cc:330] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: stopping tablet replica
I20260621 06:58:15.036345 7630 raft_consensus.cc:2229] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:15.037611 7629 tablet_replica.cc:330] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: stopping tablet replica
I20260621 06:58:15.038111 7629 raft_consensus.cc:2229] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:15.038203 7630 raft_consensus.cc:2258] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:15.038651 7629 raft_consensus.cc:2258] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:15.038931 7631 tablet_replica.cc:330] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: stopping tablet replica
I20260621 06:58:15.039433 7631 raft_consensus.cc:2229] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:15.039932 7631 raft_consensus.cc:2258] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:15.040989 7629 ts_tablet_manager.cc:1905] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:15.041826 7631 ts_tablet_manager.cc:1905] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 21 06:58:15 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:15.047971 7629 ts_tablet_manager.cc:1918] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:15.048173 7629 log.cc:1196] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/a18496dfdc144a70b108b845fceb950d
I20260621 06:58:15.048764 7629 ts_tablet_manager.cc:1939] T a18496dfdc144a70b108b845fceb950d P 6d4f3b3cea8e4c6183802add2f6aee70: Deleting consensus metadata
I20260621 06:58:15.048985 7631 ts_tablet_manager.cc:1918] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:15.049181 7631 log.cc:1196] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/a18496dfdc144a70b108b845fceb950d
I20260621 06:58:15.049782 7631 ts_tablet_manager.cc:1939] T a18496dfdc144a70b108b845fceb950d P a3dbf3b64af84a8eae9d0b2deca9d4b7: Deleting consensus metadata
I20260621 06:58:15.050733 7118 catalog_manager.cc:4841] TS 6d4f3b3cea8e4c6183802add2f6aee70 (127.2.245.194:38791): tablet a18496dfdc144a70b108b845fceb950d (table test-table [id=dde9f9b080d547b3bb19276c0f6e6e8d]) successfully deleted
I20260621 06:58:15.051723 7118 catalog_manager.cc:4841] TS a3dbf3b64af84a8eae9d0b2deca9d4b7 (127.2.245.195:35377): tablet a18496dfdc144a70b108b845fceb950d (table test-table [id=dde9f9b080d547b3bb19276c0f6e6e8d]) successfully deleted
I20260621 06:58:15.052518 7630 ts_tablet_manager.cc:1905] T a18496dfdc144a70b108b845fceb950d P 1b6c6b7e099c4c66873a5fc02f413f06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:15.054845 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.007362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.2.245.193:38859)
0621 06:58:15.007606 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:15.007615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:15.031779 (+ 24164us) server_negotiation.cc:407] Connection header received
0621 06:58:15.032400 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:15.032412 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:15.032542 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:15.032638 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:15.037130 (+ 4492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.038350 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.040730 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.041394 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.041541 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:15.046970 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:15.046985 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:15.047002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:15.047051 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:15.051707 (+ 4656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.052533 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.052540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.052543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.052741 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.053138 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.053143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.053147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.053527 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:15.053868 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:15.054174 (+ 306us) server_negotiation.cc:299] Negotiation successful
0621 06:58:15.054427 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 06:58:15.055672 7317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:15.153756 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7195
W20260621 06:58:15.171168 7120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:38859 (error 108)
I20260621 06:58:15.171571 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7328
I20260621 06:58:15.173745 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.173043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42920)
0621 06:58:15.173453 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.173549 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":223,"threads_started":1}
W20260621 06:58:15.173959 7120 catalog_manager.cc:4584] TS 1b6c6b7e099c4c66873a5fc02f413f06 (127.2.245.193:38859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a18496dfdc144a70b108b845fceb950d: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Jun 21 06:58:15 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:58:15.186471 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.172550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.2.245.194:38791)
0621 06:58:15.172724 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:15.172728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:15.172744 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:15.172861 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:15.172865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:15.172924 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:15.173048 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:15.174642 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.175413 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.176665 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.177060 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.177193 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:15.181075 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:15.181088 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:15.181104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:15.181147 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:15.183777 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.184432 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.184439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.184441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.184567 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.184955 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.184961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.184964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.185358 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:15.185643 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:15.185935 (+ 292us) server_negotiation.cc:299] Negotiation successful
0621 06:58:15.186142 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 06:58:15.187157 7450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:15.227543 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.227090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42932)
0621 06:58:15.227304 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.227405 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260621 06:58:15.228622 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.228250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42936)
0621 06:58:15.228433 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.228496 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 06:58:15.276587 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7328
I20260621 06:58:15.289263 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.288668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42952)
0621 06:58:15.288921 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.289041 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260621 06:58:15.290441 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.289958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42960)
0621 06:58:15.290155 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.290254 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260621 06:58:15.293634 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7460
Jun 21 06:58:15 dist-test-slave-9zdv krb5kdc[7090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:58:15.308060 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.294387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.2.245.195:35377)
0621 06:58:15.294559 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:15.294564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:15.294581 (+ 17us) server_negotiation.cc:407] Connection header received
0621 06:58:15.294706 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:15.294710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:15.294769 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:15.294858 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:15.296142 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.296854 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.298048 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:15.298435 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:15.298530 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:15.302745 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:15.302759 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:15.302776 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:15.302810 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:15.305591 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.306236 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.306243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.306245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.306364 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:15.306667 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:15.306670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:15.306672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:15.307085 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:15.307294 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:15.307554 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 06:58:15.307704 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260621 06:58:15.308724 7582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:15.371160 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.370579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42962)
0621 06:58:15.370820 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.370906 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260621 06:58:15.372263 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.371869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38859 (local address 127.0.0.1:42968)
0621 06:58:15.372069 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:15.372137 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260621 06:58:15.397639 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7460
I20260621 06:58:15.413939 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7103
I20260621 06:58:15.415102 7164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:15.523676 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7103
2026-06-21T06:58:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4419 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-21T06:58:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:15Z Disabled control of system clock
I20260621 06:58:15.589847 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36573
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260621 06:58:15.906277 7642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:15.906553 7642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:15.906688 7642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:15.913741 7642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:15.913910 7642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:15.914000 7642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:15.914100 7642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:15.914179 7642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:15.923223 7642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:36573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:15.924453 7642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:15.926110 7642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:15.937610 7647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:15.937639 7650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:15.937700 7648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:15.938621 7642 server_base.cc:1034] running on GCE node
I20260621 06:58:15.939414 7642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:15.940729 7642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:15.941928 7642 hybrid_clock.cc:648] HybridClock initialized: now 1782025095941891 us; error 27 us; skew 500 ppm
I20260621 06:58:15.942219 7642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:15.944305 7642 webserver.cc:460] Webserver started at http://127.2.245.254:45635/ using document root <none> and password file <none>
I20260621 06:58:15.944890 7642 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:15.944964 7642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:15.945197 7642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:15.947945 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "a3b132592c964ee7a6d56f5055927c66"
format_stamp: "Formatted at 2026-06-21 06:58:15 on dist-test-slave-9zdv"
server_key: "55c1b6b74b6820f5365641d51d0ddfb7"
server_key_iv: "ab537537191d8d81f924787b7e1318c7"
server_key_version: "encryptionkey@0"
I20260621 06:58:15.948673 7642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "a3b132592c964ee7a6d56f5055927c66"
format_stamp: "Formatted at 2026-06-21 06:58:15 on dist-test-slave-9zdv"
server_key: "55c1b6b74b6820f5365641d51d0ddfb7"
server_key_iv: "ab537537191d8d81f924787b7e1318c7"
server_key_version: "encryptionkey@0"
I20260621 06:58:15.953370 7642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260621 06:58:15.956642 7656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:15.957894 7642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 06:58:15.958029 7642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "a3b132592c964ee7a6d56f5055927c66"
format_stamp: "Formatted at 2026-06-21 06:58:15 on dist-test-slave-9zdv"
server_key: "55c1b6b74b6820f5365641d51d0ddfb7"
server_key_iv: "ab537537191d8d81f924787b7e1318c7"
server_key_version: "encryptionkey@0"
I20260621 06:58:15.958261 7642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:15.963532 7642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:15.964186 7642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:15.964427 7642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:15.981665 7642 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:36573
I20260621 06:58:15.981694 7707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:36573 every 8 connection(s)
I20260621 06:58:15.982909 7642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:15.986383 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7642
I20260621 06:58:15.986572 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:15.986749 7708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:15.987334 3031 external_mini_cluster.cc:1428] Setting key 7feb9c9d61420adf1c7c6bff3727f59d
I20260621 06:58:15.996212 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:15.989269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.2.245.254:36573)
0621 06:58:15.990323 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:15.990333 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:15.990372 (+ 39us) server_negotiation.cc:407] Connection header received
0621 06:58:15.992676 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:15.992698 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:15.993143 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:15.993540 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:15.994584 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:15.994590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:15.994606 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:15.994620 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:15.994676 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:15.994692 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:15.994926 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:15.995123 (+ 197us) server_negotiation.cc:299] Negotiation successful
0621 06:58:15.995364 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":388,"threads_started":1}
I20260621 06:58:15.999712 7708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: Bootstrap starting.
I20260621 06:58:16.002930 7708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:16.003948 7708 log.cc:824] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:16.007015 7708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: No bootstrap required, opened a new log
I20260621 06:58:16.014183 7708 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } }
I20260621 06:58:16.014555 7708 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:16.014667 7708 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b132592c964ee7a6d56f5055927c66, State: Initialized, Role: FOLLOWER
I20260621 06:58:16.015341 7708 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } }
I20260621 06:58:16.015597 7708 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:16.015753 7708 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:16.015933 7708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:16.018066 7708 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } }
I20260621 06:58:16.018544 7708 leader_election.cc:302] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3b132592c964ee7a6d56f5055927c66; no voters:
I20260621 06:58:16.019253 7708 leader_election.cc:288] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:16.019394 7713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:16.020501 7713 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [term 1 LEADER]: Becoming Leader. State: Replica: a3b132592c964ee7a6d56f5055927c66, State: Running, Role: LEADER
I20260621 06:58:16.021056 7713 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } }
I20260621 06:58:16.021358 7708 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:16.023959 7714 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3b132592c964ee7a6d56f5055927c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } } }
I20260621 06:58:16.024380 7714 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:16.024256 7715 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3b132592c964ee7a6d56f5055927c66. Latest consensus state: current_term: 1 leader_uuid: "a3b132592c964ee7a6d56f5055927c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b132592c964ee7a6d56f5055927c66" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36573 } } }
I20260621 06:58:16.024549 7715 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:16.025561 7722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:16.033171 7722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:16.041489 7722 catalog_manager.cc:1347] Generated new cluster ID: 0a7c23c8a7554fefa0c7ff76d7fa6ffc
I20260621 06:58:16.041643 7722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:16.058370 7722 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:16.058852 7722 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:16.069351 7722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: Generated new TSK 0
I20260621 06:58:16.070160 7722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:58:16.077700 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--builtin_ntp_servers=127.2.245.212:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260621 06:58:16.400490 7732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:16.400775 7732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:16.400871 7732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:16.408070 7732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:16.408169 7732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:16.408279 7732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:16.417335 7732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:16.418530 7732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:16.420200 7732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:16.433305 7737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:16.433357 7738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:16.433384 7740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:16.434357 7732 server_base.cc:1034] running on GCE node
I20260621 06:58:16.435155 7732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:16.436110 7732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:16.437498 7732 hybrid_clock.cc:648] HybridClock initialized: now 1782025096437429 us; error 54 us; skew 500 ppm
I20260621 06:58:16.437873 7732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:16.440169 7732 webserver.cc:460] Webserver started at http://127.2.245.193:39263/ using document root <none> and password file <none>
I20260621 06:58:16.440879 7732 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:16.441038 7732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:16.441361 7732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:16.444314 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "e2df8a4083584e0d959282d49d4b1f6a"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "1775f213785d2c04c35e9ca215dc12b1"
server_key_iv: "f51f3bf8b0f8ebb41a410e03dd434042"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.445384 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "e2df8a4083584e0d959282d49d4b1f6a"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "1775f213785d2c04c35e9ca215dc12b1"
server_key_iv: "f51f3bf8b0f8ebb41a410e03dd434042"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.450927 7732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 06:58:16.454528 7746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.455932 7732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:16.456113 7732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "e2df8a4083584e0d959282d49d4b1f6a"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "1775f213785d2c04c35e9ca215dc12b1"
server_key_iv: "f51f3bf8b0f8ebb41a410e03dd434042"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.456352 7732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:16.462412 7732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:16.463254 7732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:16.463521 7732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:16.464450 7732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:16.465883 7732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:16.466001 7732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.466132 7732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:16.466219 7732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.499501 7732 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:33369
I20260621 06:58:16.499552 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:33369 every 8 connection(s)
I20260621 06:58:16.500821 7732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:16.506421 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7732
I20260621 06:58:16.506616 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:16.507270 3031 external_mini_cluster.cc:1428] Setting key 3d5fd8395277062ee974b6883ff6389b
I20260621 06:58:16.509215 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:16.503450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:35627 (local address 127.2.245.254:36573)
0621 06:58:16.503840 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:16.503844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:16.505048 (+ 1204us) server_negotiation.cc:407] Connection header received
0621 06:58:16.506764 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:16.506770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:16.506844 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:16.506994 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:16.508446 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:16.508458 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:16.508462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:16.508478 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:16.508517 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:16.508529 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:16.508652 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:16.508848 (+ 196us) server_negotiation.cc:299] Negotiation successful
0621 06:58:16.508922 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":198,"threads_started":1}
I20260621 06:58:16.510090 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--builtin_ntp_servers=127.2.245.212:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260621 06:58:16.510152 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:16.504048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36573 (local address 127.2.245.193:35627)
0621 06:58:16.504894 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:16.504929 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:16.506457 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:16.507223 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:16.507231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:16.507321 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:16.508122 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:16.508126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:16.508183 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:16.508271 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:16.508625 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:16.508635 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:16.508642 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:58:16.508934 (+ 292us) client_negotiation.cc:240] Negotiation successful
0621 06:58:16.509189 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":237,"threads_started":1}
I20260621 06:58:16.511713 7859 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36573
I20260621 06:58:16.512014 7859 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:16.512575 7859 heartbeater.cc:511] Master 127.2.245.254:36573 requested a full tablet report, sending...
I20260621 06:58:16.514750 7673 ts_manager.cc:194] Registered new tserver with Master: e2df8a4083584e0d959282d49d4b1f6a (127.2.245.193:33369)
W20260621 06:58:16.825886 7863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:16.826208 7863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:16.826370 7863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:16.833664 7863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:16.833752 7863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:16.833956 7863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:16.842875 7863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:16.844203 7863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:16.845943 7863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:16.858898 7869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:16.859022 7868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:16.858989 7871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:16.860105 7863 server_base.cc:1034] running on GCE node
I20260621 06:58:16.860809 7863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:16.861959 7863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:16.863230 7863 hybrid_clock.cc:648] HybridClock initialized: now 1782025096863171 us; error 56 us; skew 500 ppm
I20260621 06:58:16.863548 7863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:16.865674 7863 webserver.cc:460] Webserver started at http://127.2.245.194:35087/ using document root <none> and password file <none>
I20260621 06:58:16.866369 7863 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:16.866473 7863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:16.866822 7863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:16.869637 7863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "90314438ed9149d2836d25eb922b9a38"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "57791eb9e56a1de66ef18375a468bbec"
server_key_iv: "f59eb550ace861f792c121e6bddce4c3"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.870461 7863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "90314438ed9149d2836d25eb922b9a38"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "57791eb9e56a1de66ef18375a468bbec"
server_key_iv: "f59eb550ace861f792c121e6bddce4c3"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.875576 7863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:58:16.879071 7877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.880465 7863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:16.880592 7863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "90314438ed9149d2836d25eb922b9a38"
format_stamp: "Formatted at 2026-06-21 06:58:16 on dist-test-slave-9zdv"
server_key: "57791eb9e56a1de66ef18375a468bbec"
server_key_iv: "f59eb550ace861f792c121e6bddce4c3"
server_key_version: "encryptionkey@0"
I20260621 06:58:16.880792 7863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:16.886122 7863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:16.886818 7863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:16.887151 7863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:16.887979 7863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:16.889276 7863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:16.889361 7863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.889441 7863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:16.889494 7863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:16.921479 7863 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:40183
I20260621 06:58:16.921566 7989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:40183 every 8 connection(s)
I20260621 06:58:16.922986 7863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:16.929605 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7863
I20260621 06:58:16.929766 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:16.930299 3031 external_mini_cluster.cc:1428] Setting key 7d533493cf4037cc44dba95f8e4291c6
I20260621 06:58:16.931180 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:16.925552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:36703 (local address 127.2.245.254:36573)
0621 06:58:16.925755 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:16.925761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:16.927227 (+ 1466us) server_negotiation.cc:407] Connection header received
0621 06:58:16.928841 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:16.928846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:16.928907 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:16.928997 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:16.930422 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:16.930427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:16.930430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:16.930445 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:16.930474 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:16.930486 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:16.930576 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:16.930804 (+ 228us) server_negotiation.cc:299] Negotiation successful
0621 06:58:16.930868 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 06:58:16.932217 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:16.926131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36573 (local address 127.2.245.194:36703)
0621 06:58:16.927086 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:16.927126 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:16.928592 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:16.929234 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:16.929245 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:16.929363 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:16.930057 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:16.930061 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:16.930109 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:16.930212 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:16.930582 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:16.930592 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:16.930599 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:58:16.930888 (+ 289us) client_negotiation.cc:240] Negotiation successful
0621 06:58:16.931156 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":288,"threads_started":1}
I20260621 06:58:16.933334 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--builtin_ntp_servers=127.2.245.212:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260621 06:58:16.933463 7990 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36573
I20260621 06:58:16.933696 7990 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:16.934274 7990 heartbeater.cc:511] Master 127.2.245.254:36573 requested a full tablet report, sending...
I20260621 06:58:16.935609 7673 ts_manager.cc:194] Registered new tserver with Master: 90314438ed9149d2836d25eb922b9a38 (127.2.245.194:40183)
W20260621 06:58:17.254426 7993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:17.254688 7993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:17.254778 7993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:17.262282 7993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:17.262374 7993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:17.262485 7993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:17.271848 7993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:17.273006 7993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:17.274737 7993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:17.287923 7999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:17.287920 8001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:17.288480 7998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:17.288754 7993 server_base.cc:1034] running on GCE node
I20260621 06:58:17.289381 7993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:17.290298 7993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:17.291522 7993 hybrid_clock.cc:648] HybridClock initialized: now 1782025097291461 us; error 77 us; skew 500 ppm
I20260621 06:58:17.291837 7993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:17.293988 7993 webserver.cc:460] Webserver started at http://127.2.245.195:46649/ using document root <none> and password file <none>
I20260621 06:58:17.294674 7993 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:17.294806 7993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:17.295197 7993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:17.298070 7993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "83d29f1b9e294f319467715f8f926557"
format_stamp: "Formatted at 2026-06-21 06:58:17 on dist-test-slave-9zdv"
server_key: "eccca06def2e747bb0dbd8de1fa8458d"
server_key_iv: "ab1a00f5f6df78f82566f24a788d4260"
server_key_version: "encryptionkey@0"
I20260621 06:58:17.298842 7993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "83d29f1b9e294f319467715f8f926557"
format_stamp: "Formatted at 2026-06-21 06:58:17 on dist-test-slave-9zdv"
server_key: "eccca06def2e747bb0dbd8de1fa8458d"
server_key_iv: "ab1a00f5f6df78f82566f24a788d4260"
server_key_version: "encryptionkey@0"
I20260621 06:58:17.303817 7993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260621 06:58:17.307474 8007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:17.308764 7993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:58:17.308888 7993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "83d29f1b9e294f319467715f8f926557"
format_stamp: "Formatted at 2026-06-21 06:58:17 on dist-test-slave-9zdv"
server_key: "eccca06def2e747bb0dbd8de1fa8458d"
server_key_iv: "ab1a00f5f6df78f82566f24a788d4260"
server_key_version: "encryptionkey@0"
I20260621 06:58:17.309078 7993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:17.314551 7993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:17.315330 7993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:17.315562 7993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:17.316378 7993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:17.317701 7993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:17.317785 7993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:17.317862 7993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:17.317916 7993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:17.350457 7993 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:41163
I20260621 06:58:17.350503 8119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:41163 every 8 connection(s)
I20260621 06:58:17.351841 7993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:17.359869 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.354467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:41657 (local address 127.2.245.254:36573)
0621 06:58:17.354688 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.354693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.356141 (+ 1448us) server_negotiation.cc:407] Connection header received
0621 06:58:17.357726 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.357730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.357783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.357870 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.359228 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.359232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.359234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.359248 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.359285 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.359295 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.359389 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.359543 (+ 154us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.359587 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 06:58:17.360917 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.355104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36573 (local address 127.2.245.195:41657)
0621 06:58:17.355986 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.356027 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.357486 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.358086 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.358094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.358189 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.358894 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.358897 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.358980 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.359077 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.359377 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.359383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.359388 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.359636 (+ 248us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.359861 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260621 06:58:17.362051 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 7993
I20260621 06:58:17.362228 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:17.362488 8120 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36573
I20260621 06:58:17.362783 8120 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:17.362836 3031 external_mini_cluster.cc:1428] Setting key c6e68a47c5045e519af1f2f435826fa7
I20260621 06:58:17.363485 8120 heartbeater.cc:511] Master 127.2.245.254:36573 requested a full tablet report, sending...
I20260621 06:58:17.365015 7673 ts_manager.cc:194] Registered new tserver with Master: 83d29f1b9e294f319467715f8f926557 (127.2.245.195:41163)
I20260621 06:58:17.365964 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:17.372727 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.370365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.2.245.254:36573)
0621 06:58:17.370550 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.370560 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.371142 (+ 582us) server_negotiation.cc:407] Connection header received
0621 06:58:17.371270 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.371274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.371322 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.371399 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.372201 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.372205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.372208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.372222 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.372251 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.372261 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.372351 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.372383 (+ 32us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.372423 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:17.376626 7672 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:17.383240 7672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:17.400590 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.394893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:41163 (local address 127.0.0.1:47266)
0621 06:58:17.397063 (+ 2170us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.397139 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.397289 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.398048 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.398058 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.398180 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.399121 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.399127 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.399168 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.399276 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.399790 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.399801 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.399809 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.400010 (+ 201us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.400105 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1898,"spinlock_wait_cycles":9856,"thread_start_us":1790,"threads_started":1}
I20260621 06:58:17.400907 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.395018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.2.245.195:41163)
0621 06:58:17.396512 (+ 1494us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.396523 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.397458 (+ 935us) server_negotiation.cc:407] Connection header received
0621 06:58:17.397540 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.397551 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.397811 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.397959 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.399443 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.399463 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.399476 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.399495 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.399555 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.399578 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.399726 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.400400 (+ 674us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.400512 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":1178,"threads_started":1}
I20260621 06:58:17.403271 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.395708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:40183 (local address 127.0.0.1:42616)
0621 06:58:17.397063 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.397139 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.397289 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.400619 (+ 3330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.400625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.400652 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.401345 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.401348 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.401364 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.401402 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.402619 (+ 1217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.402623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.402630 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.402762 (+ 132us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.402814 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"spinlock_wait_cycles":4736,"thread_start_us":925,"threads_started":1}
I20260621 06:58:17.403692 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.396351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.2.245.194:40183)
0621 06:58:17.399147 (+ 2796us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.399155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.399184 (+ 29us) server_negotiation.cc:407] Connection header received
0621 06:58:17.399252 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.399261 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.399477 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.399623 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.402248 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.402258 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.402271 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.402291 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.402351 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.402372 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.402564 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.403256 (+ 692us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.403374 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1502,"thread_start_us":1373,"threads_started":1}
I20260621 06:58:17.404561 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.397817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.2.245.193:33369)
0621 06:58:17.399606 (+ 1789us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.399617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.400178 (+ 561us) server_negotiation.cc:407] Connection header received
0621 06:58:17.400332 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.400341 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.400539 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.400677 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.403779 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.403790 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.403805 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.403827 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.403876 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.403898 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.404059 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.404153 (+ 94us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.404258 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":937,"threads_started":1}
I20260621 06:58:17.404650 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.397547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:33369 (local address 127.0.0.1:43680)
0621 06:58:17.400058 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.400073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.400203 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.401351 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.401355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.401378 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.402043 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.402046 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.402059 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.402098 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.404031 (+ 1933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.404035 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.404040 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.404173 (+ 133us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.404221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"thread_start_us":2235,"threads_started":1}
I20260621 06:58:17.405835 8055 tablet_service.cc:1467] Processing CreateTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 (DEFAULT_TABLE table=test-table [id=f3e484edb65547758cf59226e6055de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:17.407544 8055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f60e8e6623b84d37b6bd90fdf317eff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:17.407369 7925 tablet_service.cc:1467] Processing CreateTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 (DEFAULT_TABLE table=test-table [id=f3e484edb65547758cf59226e6055de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:17.407882 7794 tablet_service.cc:1467] Processing CreateTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 (DEFAULT_TABLE table=test-table [id=f3e484edb65547758cf59226e6055de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:17.408753 7925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f60e8e6623b84d37b6bd90fdf317eff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:17.408974 7794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f60e8e6623b84d37b6bd90fdf317eff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:17.420318 8138 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Bootstrap starting.
I20260621 06:58:17.423656 8138 tablet_bootstrap.cc:654] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:17.425034 8138 log.cc:824] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:17.428471 8139 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Bootstrap starting.
I20260621 06:58:17.429454 8140 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Bootstrap starting.
I20260621 06:58:17.432201 8139 tablet_bootstrap.cc:654] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:17.433651 8139 log.cc:824] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:17.433733 8138 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: No bootstrap required, opened a new log
I20260621 06:58:17.434092 8138 ts_tablet_manager.cc:1397] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260621 06:58:17.434171 8140 tablet_bootstrap.cc:654] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:17.435365 8140 log.cc:824] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:17.436817 8139 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: No bootstrap required, opened a new log
I20260621 06:58:17.437070 8139 ts_tablet_manager.cc:1397] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260621 06:58:17.438052 8140 tablet_bootstrap.cc:492] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: No bootstrap required, opened a new log
I20260621 06:58:17.438288 8140 ts_tablet_manager.cc:1397] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260621 06:58:17.443895 8139 raft_consensus.cc:348] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.444471 8139 raft_consensus.cc:374] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:17.444638 8139 raft_consensus.cc:729] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d29f1b9e294f319467715f8f926557, State: Initialized, Role: FOLLOWER
I20260621 06:58:17.444446 8138 raft_consensus.cc:348] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.444959 8138 raft_consensus.cc:374] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:17.445086 8138 raft_consensus.cc:729] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2df8a4083584e0d959282d49d4b1f6a, State: Initialized, Role: FOLLOWER
I20260621 06:58:17.445420 8139 consensus_queue.cc:260] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.445657 8138 consensus_queue.cc:260] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.446544 8120 heartbeater.cc:503] Master 127.2.245.254:36573 was elected leader, sending a full tablet report...
I20260621 06:58:17.446986 7859 heartbeater.cc:503] Master 127.2.245.254:36573 was elected leader, sending a full tablet report...
I20260621 06:58:17.447964 8138 ts_tablet_manager.cc:1428] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260621 06:58:17.448647 8139 ts_tablet_manager.cc:1428] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260621 06:58:17.448303 8140 raft_consensus.cc:348] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.449209 8140 raft_consensus.cc:374] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:17.449324 8140 raft_consensus.cc:729] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90314438ed9149d2836d25eb922b9a38, State: Initialized, Role: FOLLOWER
I20260621 06:58:17.449873 8140 consensus_queue.cc:260] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.451457 8140 ts_tablet_manager.cc:1428] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260621 06:58:17.451736 7990 heartbeater.cc:503] Master 127.2.245.254:36573 was elected leader, sending a full tablet report...
W20260621 06:58:17.503074 7860 tablet.cc:2369] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:17.603585 8121 tablet.cc:2369] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:17.675125 7991 tablet.cc:2369] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:58:17.743757 8146 raft_consensus.cc:482] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:17.744014 8146 raft_consensus.cc:504] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.745639 8146 leader_election.cc:288] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83d29f1b9e294f319467715f8f926557 (127.2.245.195:41163), e2df8a4083584e0d959282d49d4b1f6a (127.2.245.193:33369)
I20260621 06:58:17.749246 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.746221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:53969 (local address 127.2.245.195:41163)
0621 06:58:17.746470 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.746479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.746925 (+ 446us) server_negotiation.cc:407] Connection header received
0621 06:58:17.747160 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.747169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.747253 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.747361 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.748547 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.748553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.748559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.748578 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.748616 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.748634 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.748754 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.748868 (+ 114us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.748963 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 06:58:17.749297 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.746082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:41163 (local address 127.2.245.194:53969)
0621 06:58:17.746755 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.746770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.746956 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.747419 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.747424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.747447 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.748275 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.748279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.748304 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.748355 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.748744 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.748751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.748760 (+ 9us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.748863 (+ 103us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.748916 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":379,"threads_started":1}
I20260621 06:58:17.749643 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.747095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:42271 (local address 127.2.245.193:33369)
0621 06:58:17.747298 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.747306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.747701 (+ 395us) server_negotiation.cc:407] Connection header received
0621 06:58:17.747863 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.747868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.747963 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.748071 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.748979 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.748983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.748988 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.749003 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.749036 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.749051 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.749158 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.749287 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.749341 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260621 06:58:17.749756 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.746916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:33369 (local address 127.2.245.194:42271)
0621 06:58:17.747573 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:17.747596 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:17.747719 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:17.748129 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:17.748132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:17.748169 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:17.748798 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:58:17.748803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:17.748819 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:58:17.748853 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:17.749155 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:17.749159 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:17.749165 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 06:58:17.749312 (+ 147us) client_negotiation.cc:240] Negotiation successful
0621 06:58:17.749355 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":291,"threads_started":1}
I20260621 06:58:17.750325 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f60e8e6623b84d37b6bd90fdf317eff6" candidate_uuid: "90314438ed9149d2836d25eb922b9a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d29f1b9e294f319467715f8f926557" is_pre_election: true
I20260621 06:58:17.750594 7814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f60e8e6623b84d37b6bd90fdf317eff6" candidate_uuid: "90314438ed9149d2836d25eb922b9a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2df8a4083584e0d959282d49d4b1f6a" is_pre_election: true
I20260621 06:58:17.750790 8075 raft_consensus.cc:2454] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90314438ed9149d2836d25eb922b9a38 in term 0.
I20260621 06:58:17.751101 7814 raft_consensus.cc:2454] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90314438ed9149d2836d25eb922b9a38 in term 0.
I20260621 06:58:17.751489 7879 leader_election.cc:302] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83d29f1b9e294f319467715f8f926557, 90314438ed9149d2836d25eb922b9a38; no voters:
I20260621 06:58:17.751839 8146 raft_consensus.cc:2781] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:17.752012 8146 raft_consensus.cc:482] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:17.752115 8146 raft_consensus.cc:3037] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:17.754318 8146 raft_consensus.cc:504] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.754856 8146 leader_election.cc:288] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [CANDIDATE]: Term 1 election: Requested vote from peers 83d29f1b9e294f319467715f8f926557 (127.2.245.195:41163), e2df8a4083584e0d959282d49d4b1f6a (127.2.245.193:33369)
I20260621 06:58:17.755333 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f60e8e6623b84d37b6bd90fdf317eff6" candidate_uuid: "90314438ed9149d2836d25eb922b9a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d29f1b9e294f319467715f8f926557"
I20260621 06:58:17.755415 7814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f60e8e6623b84d37b6bd90fdf317eff6" candidate_uuid: "90314438ed9149d2836d25eb922b9a38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2df8a4083584e0d959282d49d4b1f6a"
I20260621 06:58:17.755517 8075 raft_consensus.cc:3037] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:17.755658 7814 raft_consensus.cc:3037] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:17.757843 7814 raft_consensus.cc:2454] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90314438ed9149d2836d25eb922b9a38 in term 1.
I20260621 06:58:17.757872 8075 raft_consensus.cc:2454] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90314438ed9149d2836d25eb922b9a38 in term 1.
I20260621 06:58:17.758213 7879 leader_election.cc:302] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90314438ed9149d2836d25eb922b9a38, e2df8a4083584e0d959282d49d4b1f6a; no voters:
I20260621 06:58:17.758543 8146 raft_consensus.cc:2781] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:17.759100 8146 raft_consensus.cc:686] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 1 LEADER]: Becoming Leader. State: Replica: 90314438ed9149d2836d25eb922b9a38, State: Running, Role: LEADER
I20260621 06:58:17.759552 8146 consensus_queue.cc:237] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } }
I20260621 06:58:17.764242 7672 catalog_manager.cc:5495] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90314438ed9149d2836d25eb922b9a38 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "90314438ed9149d2836d25eb922b9a38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90314438ed9149d2836d25eb922b9a38" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40183 } health_report { overall_health: HEALTHY } } }
I20260621 06:58:17.856359 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.854189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.2.245.194:40183)
0621 06:58:17.854372 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.854376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.854394 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:58:17.854608 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.854613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.854725 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.854840 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.855781 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.855785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.855788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.855800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.855830 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.855841 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.855936 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.856023 (+ 87us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.856066 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260621 06:58:17.865110 8075 raft_consensus.cc:1261] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 1 FOLLOWER]: Refusing update from remote peer 90314438ed9149d2836d25eb922b9a38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:17.865124 7814 raft_consensus.cc:1261] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 1 FOLLOWER]: Refusing update from remote peer 90314438ed9149d2836d25eb922b9a38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:17.866130 8149 consensus_queue.cc:1035] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83d29f1b9e294f319467715f8f926557" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:17.866518 8146 consensus_queue.cc:1035] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2df8a4083584e0d959282d49d4b1f6a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:17.877724 8152 mvcc.cc:205] Tried to move back new op lower bound from 7299174800844529664 to 7299174800426831872. Current Snapshot: MvccSnapshot[applied={T|T < 7299174800844529664}]
I20260621 06:58:17.881220 8153 mvcc.cc:205] Tried to move back new op lower bound from 7299174800844529664 to 7299174800426831872. Current Snapshot: MvccSnapshot[applied={T|T < 7299174800844529664}]
I20260621 06:58:17.890584 7673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:17.890880 7673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:17.894263 7673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3b132592c964ee7a6d56f5055927c66: Sending DeleteTablet for 3 replicas of tablet f60e8e6623b84d37b6bd90fdf317eff6
I20260621 06:58:17.895444 8055 tablet_service.cc:1514] Processing DeleteTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:17 UTC) from {username='slave'} at 127.0.0.1:47266
I20260621 06:58:17.895860 7925 tablet_service.cc:1514] Processing DeleteTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:17 UTC) from {username='slave'} at 127.0.0.1:42616
I20260621 06:58:17.896199 7794 tablet_service.cc:1514] Processing DeleteTablet for tablet f60e8e6623b84d37b6bd90fdf317eff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:17 UTC) from {username='slave'} at 127.0.0.1:43680
I20260621 06:58:17.896328 8161 tablet_replica.cc:330] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: stopping tablet replica
I20260621 06:58:17.896754 8161 raft_consensus.cc:2229] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:17.897691 8161 raft_consensus.cc:2258] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:17.898093 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7732
I20260621 06:58:17.904856 8161 ts_tablet_manager.cc:1905] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:17.908998 8163 tablet_replica.cc:330] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: stopping tablet replica
I20260621 06:58:17.909564 8163 raft_consensus.cc:2229] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:17.910184 8163 raft_consensus.cc:2258] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:17.911249 8162 tablet_replica.cc:330] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: stopping tablet replica
I20260621 06:58:17.911847 8162 raft_consensus.cc:2229] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:17.912842 8162 raft_consensus.cc:2258] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:17.914925 8163 ts_tablet_manager.cc:1905] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:17.915525 8162 ts_tablet_manager.cc:1905] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:17.917582 8161 ts_tablet_manager.cc:1918] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:17.924167 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:17.899286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.2.245.193:33369)
0621 06:58:17.899459 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:17.899465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:17.921607 (+ 22142us) server_negotiation.cc:407] Connection header received
0621 06:58:17.921859 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:17.921865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:17.921967 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:17.922082 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:17.923341 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:17.923348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:17.923352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:17.923374 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:17.923413 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:17.923430 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:17.923541 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:17.923793 (+ 252us) server_negotiation.cc:299] Negotiation successful
0621 06:58:17.923862 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:58:17.925529 7854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:17.925843 8162 ts_tablet_manager.cc:1918] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:17.926025 8162 log.cc:1196] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/f60e8e6623b84d37b6bd90fdf317eff6
I20260621 06:58:17.926519 8162 ts_tablet_manager.cc:1939] T f60e8e6623b84d37b6bd90fdf317eff6 P 90314438ed9149d2836d25eb922b9a38: Deleting consensus metadata
I20260621 06:58:17.928121 7657 catalog_manager.cc:4841] TS 90314438ed9149d2836d25eb922b9a38 (127.2.245.194:40183): tablet f60e8e6623b84d37b6bd90fdf317eff6 (table test-table [id=f3e484edb65547758cf59226e6055de2]) successfully deleted
I20260621 06:58:17.931210 8161 log.cc:1196] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/f60e8e6623b84d37b6bd90fdf317eff6
I20260621 06:58:17.932044 8161 ts_tablet_manager.cc:1939] T f60e8e6623b84d37b6bd90fdf317eff6 P 83d29f1b9e294f319467715f8f926557: Deleting consensus metadata
I20260621 06:58:17.936672 7658 catalog_manager.cc:4841] TS 83d29f1b9e294f319467715f8f926557 (127.2.245.195:41163): tablet f60e8e6623b84d37b6bd90fdf317eff6 (table test-table [id=f3e484edb65547758cf59226e6055de2]) successfully deleted
I20260621 06:58:18.040786 8163 ts_tablet_manager.cc:1918] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:18.040997 8163 log.cc:1196] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/f60e8e6623b84d37b6bd90fdf317eff6
I20260621 06:58:18.041451 8163 ts_tablet_manager.cc:1939] T f60e8e6623b84d37b6bd90fdf317eff6 P e2df8a4083584e0d959282d49d4b1f6a: Deleting consensus metadata
I20260621 06:58:18.043088 7658 catalog_manager.cc:4841] TS e2df8a4083584e0d959282d49d4b1f6a (127.2.245.193:33369): tablet f60e8e6623b84d37b6bd90fdf317eff6 (table test-table [id=f3e484edb65547758cf59226e6055de2]) successfully deleted
I20260621 06:58:18.043258 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7732
I20260621 06:58:18.065367 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7863
I20260621 06:58:18.068338 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:18.066305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.2.245.194:40183)
0621 06:58:18.066465 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:18.066470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:18.066493 (+ 23us) server_negotiation.cc:407] Connection header received
0621 06:58:18.066594 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:18.066598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:18.066648 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:18.066734 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:18.067680 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:18.067685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:18.067688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:18.067703 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:18.067734 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:18.067745 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:18.067852 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:18.067972 (+ 120us) server_negotiation.cc:299] Negotiation successful
0621 06:58:18.068018 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 06:58:18.068998 7985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:18.179181 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7863
I20260621 06:58:18.197510 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7993
I20260621 06:58:18.200517 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:18.198255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.2.245.195:41163)
0621 06:58:18.198440 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:18.198459 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:18.198546 (+ 87us) server_negotiation.cc:407] Connection header received
0621 06:58:18.198768 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:18.198772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:18.198861 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:18.199041 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:18.199858 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:18.199863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:18.199868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:18.199884 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:18.199921 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:18.199932 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:18.200071 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:18.200148 (+ 77us) server_negotiation.cc:299] Negotiation successful
0621 06:58:18.200205 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 06:58:18.201152 8115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:18.288179 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7993
I20260621 06:58:18.303820 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 7642
I20260621 06:58:18.305076 7703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:18.408505 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 7642
2026-06-21T06:58:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2882 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260621 06:58:18.448381 3031 mini_oidc.cc:152] Starting JWKS server
I20260621 06:58:18.450999 3031 webserver.cc:460] Webserver started at https://127.0.0.1:37957/ using document root <none> and password file <none>
I20260621 06:58:18.451386 3031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 06:58:18.452037 3031 webserver.cc:460] Webserver started at http://127.0.0.1:46223/ using document root <none> and password file <none>
2026-06-21T06:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:18Z Disabled control of system clock
I20260621 06:58:18.478606 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:45435
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:45435
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37957/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/testchainca.cert with env {}
W20260621 06:58:18.796007 8174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:18.796281 8174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:18.796365 8174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:18.803572 8174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:18.803679 8174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:18.803726 8174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:18.803781 8174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:18.803826 8174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 06:58:18.803880 8174 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 06:58:18.803920 8174 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37957/jwks.json
I20260621 06:58:18.813674 8174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:45435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:45435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37957/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:18.815008 8174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:18.816263 8174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:18.827841 8180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:18.827877 8182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:18.828358 8179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:18.828744 8174 server_base.cc:1034] running on GCE node
I20260621 06:58:18.829403 8174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:18.830752 8174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:18.831951 8174 hybrid_clock.cc:648] HybridClock initialized: now 1782025098831889 us; error 54 us; skew 500 ppm
I20260621 06:58:18.832276 8174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:18.834486 8174 webserver.cc:460] Webserver started at http://127.2.245.254:35817/ using document root <none> and password file <none>
I20260621 06:58:18.835278 8174 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:18.835366 8174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:18.835646 8174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:18.838380 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "ff0e03a4d9db46209b182b0380e775fa"
format_stamp: "Formatted at 2026-06-21 06:58:18 on dist-test-slave-9zdv"
server_key: "047f52bfd4069737f201e8148dd296fa"
server_key_iv: "8f7d1624e9336aefd08dbba25a519f01"
server_key_version: "encryptionkey@0"
I20260621 06:58:18.839154 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "ff0e03a4d9db46209b182b0380e775fa"
format_stamp: "Formatted at 2026-06-21 06:58:18 on dist-test-slave-9zdv"
server_key: "047f52bfd4069737f201e8148dd296fa"
server_key_iv: "8f7d1624e9336aefd08dbba25a519f01"
server_key_version: "encryptionkey@0"
I20260621 06:58:18.844151 8174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:58:18.847517 8188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:18.848896 8174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260621 06:58:18.849038 8174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "ff0e03a4d9db46209b182b0380e775fa"
format_stamp: "Formatted at 2026-06-21 06:58:18 on dist-test-slave-9zdv"
server_key: "047f52bfd4069737f201e8148dd296fa"
server_key_iv: "8f7d1624e9336aefd08dbba25a519f01"
server_key_version: "encryptionkey@0"
I20260621 06:58:18.849211 8174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:18.860122 8174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:18.860821 8174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:18.861061 8174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:18.878207 8174 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:45435
I20260621 06:58:18.878234 8239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:45435 every 8 connection(s)
I20260621 06:58:18.879487 8174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:18.883335 8240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:18.886319 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8174
I20260621 06:58:18.886509 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:18.887197 3031 external_mini_cluster.cc:1428] Setting key 2e557895fe2cbd1dd82bc23ea7f8bcd0
I20260621 06:58:18.896849 8240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa: Bootstrap starting.
I20260621 06:58:18.900331 8240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:18.901932 8240 log.cc:824] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:18.905493 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:18.889056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.2.245.254:45435)
0621 06:58:18.890072 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:18.890086 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:18.890137 (+ 51us) server_negotiation.cc:407] Connection header received
0621 06:58:18.892083 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:18.892116 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:18.892746 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:18.893171 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:18.893699 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:18.895771 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:18.898546 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:18.899617 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:18.899812 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:18.901405 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:18.901418 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:18.901440 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:18.902408 (+ 968us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:18.902545 (+ 137us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:18.902576 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:18.902712 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:18.903403 (+ 691us) server_negotiation.cc:299] Negotiation successful
0621 06:58:18.903950 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":285,"threads_started":1}
I20260621 06:58:18.905890 8240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa: No bootstrap required, opened a new log
I20260621 06:58:18.910151 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 06:58:18.917379 8240 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } }
I20260621 06:58:18.917943 8240 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:18.918134 8240 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff0e03a4d9db46209b182b0380e775fa, State: Initialized, Role: FOLLOWER
I20260621 06:58:18.918834 8240 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } }
I20260621 06:58:18.919158 8240 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:18.919304 8240 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:18.919463 8240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:18.923308 8240 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } }
I20260621 06:58:18.924019 8240 leader_election.cc:302] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff0e03a4d9db46209b182b0380e775fa; no voters:
I20260621 06:58:18.924649 8240 leader_election.cc:288] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:18.924809 8245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:18.925504 8245 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [term 1 LEADER]: Becoming Leader. State: Replica: ff0e03a4d9db46209b182b0380e775fa, State: Running, Role: LEADER
I20260621 06:58:18.926028 8245 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } }
I20260621 06:58:18.926806 8240 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:18.928915 8246 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff0e03a4d9db46209b182b0380e775fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } } }
I20260621 06:58:18.929386 8246 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:18.928887 8247 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff0e03a4d9db46209b182b0380e775fa. Latest consensus state: current_term: 1 leader_uuid: "ff0e03a4d9db46209b182b0380e775fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0e03a4d9db46209b182b0380e775fa" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 45435 } } }
I20260621 06:58:18.935161 8247 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:18.942122 8250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:18.947147 8250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:18.959800 8250 catalog_manager.cc:1347] Generated new cluster ID: 25a9137a906b491aae21405d396afc7c
I20260621 06:58:18.959957 8250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:18.981724 8250 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:18.983001 8250 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:18.989516 8250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff0e03a4d9db46209b182b0380e775fa: Generated new TSK 0
I20260621 06:58:18.990546 8250 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260621 06:58:19.035843 8270 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260621 06:58:19.036013 8270 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260621 06:58:19.036769 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:19.008080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.2.245.254:45435)
0621 06:58:19.008333 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:19.008339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:19.008787 (+ 448us) server_negotiation.cc:407] Connection header received
0621 06:58:19.009088 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:19.009092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:19.009171 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:19.009285 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0621 06:58:19.009880 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.010627 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.012721 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.013127 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.013221 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:19.013804 (+ 583us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 06:58:19.036249 (+ 22445us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0621 06:58:19.036535 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54182: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260621 06:58:19.036914 8243 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54182: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260621 06:58:19.038690 8269 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:19.007948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:45435 (local address 127.0.0.1:54182)
0621 06:58:19.008655 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:19.008675 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:19.008880 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:19.009351 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:19.009356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:19.009368 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0621 06:58:19.009743 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:19.009754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.010778 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.010784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:19.012567 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:19.012584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.013278 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.013284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:19.013471 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:19.013544 (+ 73us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 06:58:19.036563 (+ 23019us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0621 06:58:19.036919 (+ 356us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.245.254:45435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":337,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.245.254:45435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37957/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260621 06:58:19.040915 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8174
I20260621 06:58:19.041989 8235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:19.133167 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8174
2026-06-21T06:58:19Z chronyd exiting
I20260621 06:58:19.183089 3031 test_util.cc:175] -----------------------------------------------
I20260621 06:58:19.183251 3031 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782025057166075-3031-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260621 06:58:19.186813 3031 mini_oidc.cc:152] Starting JWKS server
I20260621 06:58:19.188663 3031 webserver.cc:460] Webserver started at https://127.0.0.1:41231/ using document root <none> and password file <none>
I20260621 06:58:19.188889 3031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 06:58:19.189558 3031 webserver.cc:460] Webserver started at http://127.0.0.1:41885/ using document root <none> and password file <none>
2026-06-21T06:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:19Z Disabled control of system clock
I20260621 06:58:19.215854 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33123
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:33123
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41231/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/testchainca.cert with env {}
W20260621 06:58:19.533524 8277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:19.533811 8277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:19.533897 8277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:19.541126 8277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:19.541234 8277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:19.541270 8277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:19.541317 8277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:19.541361 8277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 06:58:19.541415 8277 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 06:58:19.541453 8277 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41231/jwks.json
I20260621 06:58:19.551822 8277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:33123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41231/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:19.553066 8277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:19.554318 8277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:19.565927 8282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:19.565907 8285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:19.566044 8283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:19.566869 8277 server_base.cc:1034] running on GCE node
I20260621 06:58:19.567637 8277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:19.568917 8277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:19.570111 8277 hybrid_clock.cc:648] HybridClock initialized: now 1782025099570066 us; error 44 us; skew 500 ppm
I20260621 06:58:19.570436 8277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:19.572705 8277 webserver.cc:460] Webserver started at http://127.2.245.254:42681/ using document root <none> and password file <none>
I20260621 06:58:19.573359 8277 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:19.573443 8277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:19.573782 8277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:19.576610 8277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "4d218866dfd74b9c8e0f7b3d7e75620f"
format_stamp: "Formatted at 2026-06-21 06:58:19 on dist-test-slave-9zdv"
server_key: "0f1a56f036ad4f3d36cbf712fd1c79bd"
server_key_iv: "0e8b2ab74e0e870f11e1537cc6e2b30c"
server_key_version: "encryptionkey@0"
I20260621 06:58:19.577405 8277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "4d218866dfd74b9c8e0f7b3d7e75620f"
format_stamp: "Formatted at 2026-06-21 06:58:19 on dist-test-slave-9zdv"
server_key: "0f1a56f036ad4f3d36cbf712fd1c79bd"
server_key_iv: "0e8b2ab74e0e870f11e1537cc6e2b30c"
server_key_version: "encryptionkey@0"
I20260621 06:58:19.582244 8277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260621 06:58:19.585671 8291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:19.587050 8277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:19.587231 8277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "4d218866dfd74b9c8e0f7b3d7e75620f"
format_stamp: "Formatted at 2026-06-21 06:58:19 on dist-test-slave-9zdv"
server_key: "0f1a56f036ad4f3d36cbf712fd1c79bd"
server_key_iv: "0e8b2ab74e0e870f11e1537cc6e2b30c"
server_key_version: "encryptionkey@0"
I20260621 06:58:19.587452 8277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:19.603464 8277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:19.604311 8277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:19.604619 8277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:19.622398 8277 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:33123
I20260621 06:58:19.622438 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:33123 every 8 connection(s)
I20260621 06:58:19.623835 8277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:19.627385 8343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:19.633935 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8277
I20260621 06:58:19.634161 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:19.634788 3031 external_mini_cluster.cc:1428] Setting key 25307cda1c8765171ce1dd38d7365397
I20260621 06:58:19.637687 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f: Bootstrap starting.
I20260621 06:58:19.640786 8343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:19.641870 8343 log.cc:824] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:19.644668 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f: No bootstrap required, opened a new log
I20260621 06:58:19.647857 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:19.636659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.2.245.254:33123)
0621 06:58:19.637594 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:19.637604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:19.637644 (+ 40us) server_negotiation.cc:407] Connection header received
0621 06:58:19.639398 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:19.639419 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:19.639852 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:19.640224 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:19.640753 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.642148 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.643542 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.644231 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.644414 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:19.645598 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:19.645605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:19.645620 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:19.646135 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:19.646212 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:19.646233 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:19.646367 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:19.646581 (+ 214us) server_negotiation.cc:299] Negotiation successful
0621 06:58:19.646864 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":325,"threads_started":1}
I20260621 06:58:19.651649 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 06:58:19.652664 8343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } }
I20260621 06:58:19.653041 8343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:19.653149 8343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d218866dfd74b9c8e0f7b3d7e75620f, State: Initialized, Role: FOLLOWER
I20260621 06:58:19.653779 8343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } }
I20260621 06:58:19.654121 8343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:19.654282 8343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:19.654486 8343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:19.656872 8343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } }
I20260621 06:58:19.657318 8343 leader_election.cc:302] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d218866dfd74b9c8e0f7b3d7e75620f; no voters:
I20260621 06:58:19.657847 8343 leader_election.cc:288] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:19.658013 8348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:19.658713 8348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [term 1 LEADER]: Becoming Leader. State: Replica: 4d218866dfd74b9c8e0f7b3d7e75620f, State: Running, Role: LEADER
I20260621 06:58:19.659268 8348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } }
I20260621 06:58:19.660070 8343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:19.662412 8349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } } }
I20260621 06:58:19.662501 8350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d218866dfd74b9c8e0f7b3d7e75620f. Latest consensus state: current_term: 1 leader_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d218866dfd74b9c8e0f7b3d7e75620f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33123 } } }
I20260621 06:58:19.662993 8349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:19.662993 8350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:19.663888 8355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:19.669018 8355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:19.688923 8355 catalog_manager.cc:1347] Generated new cluster ID: 31bf58004cff484581a99d43863626c8
I20260621 06:58:19.689056 8355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:19.705746 8355 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:19.707598 8355 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:19.725739 8355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d218866dfd74b9c8e0f7b3d7e75620f: Generated new TSK 0
I20260621 06:58:19.726620 8355 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260621 06:58:19.762200 8373 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260621 06:58:19.762327 8373 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260621 06:58:19.763558 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:19.743286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.2.245.254:33123)
0621 06:58:19.743450 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:19.743460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:19.743809 (+ 349us) server_negotiation.cc:407] Connection header received
0621 06:58:19.744014 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:19.744021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:19.744123 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:19.744287 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0621 06:58:19.744750 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.745921 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.747322 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.748016 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.748141 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:19.748640 (+ 499us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 06:58:19.762856 (+ 14216us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0621 06:58:19.763258 (+ 402us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260621 06:58:19.763579 8372 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:19.743143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:33123 (local address 127.0.0.1:49330)
0621 06:58:19.743715 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:19.743730 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:19.743841 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:19.744314 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:19.744317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:19.744326 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0621 06:58:19.744560 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:19.744567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.746076 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.746080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:19.747193 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:19.747205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:19.748211 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:19.748215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:19.748353 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:19.748385 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 06:58:19.763102 (+ 14717us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0621 06:58:19.763298 (+ 196us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.245.254:33123: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":308,"threads_started":1}
W20260621 06:58:19.763677 8346 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260621 06:58:19.764948 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8277
I20260621 06:58:19.766057 8338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:19.861176 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8277
2026-06-21T06:58:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (728 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260621 06:58:19.915889 3031 mini_oidc.cc:152] Starting JWKS server
I20260621 06:58:19.917685 3031 webserver.cc:460] Webserver started at https://127.0.0.1:33407/ using document root <none> and password file <none>
I20260621 06:58:19.917936 3031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 06:58:19.918560 3031 webserver.cc:460] Webserver started at http://127.0.0.1:36865/ using document root <none> and password file <none>
2026-06-21T06:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:19Z Disabled control of system clock
I20260621 06:58:19.945758 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46255
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:33923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:46255
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33407/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260621 06:58:20.259828 8380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:20.260120 8380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:20.260316 8380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:20.267514 8380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:20.267611 8380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:20.267736 8380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:20.267832 8380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:20.267899 8380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 06:58:20.267987 8380 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 06:58:20.268059 8380 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33407/jwks.json
I20260621 06:58:20.277254 8380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:33923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:46255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33407/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:20.278575 8380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:20.280709 8380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:20.295471 8385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:20.295473 8386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:20.295603 8388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:20.296111 8380 server_base.cc:1034] running on GCE node
I20260621 06:58:20.297140 8380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:20.298902 8380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:20.300181 8380 hybrid_clock.cc:648] HybridClock initialized: now 1782025100300141 us; error 35 us; skew 500 ppm
I20260621 06:58:20.300477 8380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:20.302729 8380 webserver.cc:460] Webserver started at http://127.2.245.254:42147/ using document root <none> and password file <none>
I20260621 06:58:20.303432 8380 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:20.303515 8380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:20.303773 8380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:20.306509 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "25057eee338a4ad09752a807c382cab8"
format_stamp: "Formatted at 2026-06-21 06:58:20 on dist-test-slave-9zdv"
server_key: "8ab8a6b6857cb90fd09c75760d882bdc"
server_key_iv: "09d4842d5233a9107db5978d836850aa"
server_key_version: "encryptionkey@0"
I20260621 06:58:20.307318 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "25057eee338a4ad09752a807c382cab8"
format_stamp: "Formatted at 2026-06-21 06:58:20 on dist-test-slave-9zdv"
server_key: "8ab8a6b6857cb90fd09c75760d882bdc"
server_key_iv: "09d4842d5233a9107db5978d836850aa"
server_key_version: "encryptionkey@0"
I20260621 06:58:20.312279 8380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 06:58:20.315726 8394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:20.317010 8380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:20.317212 8380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "25057eee338a4ad09752a807c382cab8"
format_stamp: "Formatted at 2026-06-21 06:58:20 on dist-test-slave-9zdv"
server_key: "8ab8a6b6857cb90fd09c75760d882bdc"
server_key_iv: "09d4842d5233a9107db5978d836850aa"
server_key_version: "encryptionkey@0"
I20260621 06:58:20.317420 8380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:20.340889 8380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:20.341765 8380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:20.342072 8380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:20.360478 8380 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:46255
I20260621 06:58:20.360486 8445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:46255 every 8 connection(s)
I20260621 06:58:20.361847 8380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:20.363706 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8380
I20260621 06:58:20.363924 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:20.364583 3031 external_mini_cluster.cc:1428] Setting key a0928c9caf569325fab65f5c27a201f6
I20260621 06:58:20.366219 8446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:20.379914 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:20.366563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.2.245.254:46255)
0621 06:58:20.367656 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:20.367672 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:20.367724 (+ 52us) server_negotiation.cc:407] Connection header received
0621 06:58:20.369611 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:20.369638 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:20.370228 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:20.370696 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:20.371257 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.373300 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.374649 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.375695 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.375870 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:20.376986 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:20.376997 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:20.377016 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:20.377790 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:20.377890 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:20.377920 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:20.378046 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:20.378263 (+ 217us) server_negotiation.cc:299] Negotiation successful
0621 06:58:20.378616 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":359,"threads_started":1}
I20260621 06:58:20.380838 8446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8: Bootstrap starting.
I20260621 06:58:20.383625 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 06:58:20.384325 8446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:20.385558 8446 log.cc:824] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:20.388576 8446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8: No bootstrap required, opened a new log
I20260621 06:58:20.395759 8446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } }
I20260621 06:58:20.396087 8446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:20.396169 8446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25057eee338a4ad09752a807c382cab8, State: Initialized, Role: FOLLOWER
I20260621 06:58:20.396726 8446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } }
I20260621 06:58:20.396919 8446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:20.397050 8446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:20.397228 8446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:20.399571 8446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } }
I20260621 06:58:20.400020 8446 leader_election.cc:302] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25057eee338a4ad09752a807c382cab8; no voters:
I20260621 06:58:20.400525 8446 leader_election.cc:288] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:20.400722 8451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:20.401455 8451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [term 1 LEADER]: Becoming Leader. State: Replica: 25057eee338a4ad09752a807c382cab8, State: Running, Role: LEADER
I20260621 06:58:20.401983 8451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } }
I20260621 06:58:20.402597 8446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:20.404906 8452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25057eee338a4ad09752a807c382cab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } } }
I20260621 06:58:20.404915 8453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25057eee338a4ad09752a807c382cab8. Latest consensus state: current_term: 1 leader_uuid: "25057eee338a4ad09752a807c382cab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25057eee338a4ad09752a807c382cab8" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46255 } } }
I20260621 06:58:20.405319 8452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:20.405323 8453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:20.408195 8457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:20.413933 8457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:20.422809 8457 catalog_manager.cc:1347] Generated new cluster ID: a949e1af86e14df287b3f7334c96d207
I20260621 06:58:20.422923 8457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:20.443707 8457 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:20.444979 8457 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:20.457568 8457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25057eee338a4ad09752a807c382cab8: Generated new TSK 0
I20260621 06:58:20.458650 8457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:58:20.483635 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:20.475172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.2.245.254:46255)
0621 06:58:20.475378 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:20.475385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:20.475819 (+ 434us) server_negotiation.cc:407] Connection header received
0621 06:58:20.476014 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:20.476022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:20.476123 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:20.476295 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0621 06:58:20.476728 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.477889 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.479310 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.480024 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.480159 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:20.480657 (+ 498us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 06:58:20.482971 (+ 2314us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0621 06:58:20.483294 (+ 323us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46282: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260621 06:58:20.483790 8449 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46282: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260621 06:58:20.485416 8475 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:20.475045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46255 (local address 127.0.0.1:46282)
0621 06:58:20.475716 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:20.475732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:20.475851 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:20.476313 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:20.476316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:20.476326 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0621 06:58:20.476563 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:20.476571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.478033 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.478037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:20.479165 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:20.479177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:20.480230 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:20.480234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:20.480361 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:20.480392 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 06:58:20.483211 (+ 2819us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0621 06:58:20.484987 (+ 1776us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.245.254:46255: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":1489,"thread_start_us":361,"threads_started":1}
I20260621 06:58:20.486361 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8380
I20260621 06:58:20.487635 8441 generic_service.cc:196] Checking for leaks (request via RPC)
E20260621 06:58:20.499787 8476 webserver.cc:533] Webserver: sslize failed:
E20260621 06:58:20.499935 8476 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260621 06:58:20.580780 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8380
2026-06-21T06:58:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (720 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-21T06:58:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:20Z Disabled control of system clock
I20260621 06:58:20.662849 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:39331
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:39331
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/testchainca.cert with env {}
W20260621 06:58:20.986645 8481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:20.986963 8481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:20.987105 8481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:20.994354 8481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:20.994467 8481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:20.994504 8481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:20.994560 8481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:20.994604 8481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 06:58:20.994659 8481 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 06:58:20.994701 8481 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260621 06:58:21.004704 8481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:39331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:21.005945 8481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:21.007264 8481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:21.019065 8486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:21.019200 8489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:21.019217 8487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:21.020289 8481 server_base.cc:1034] running on GCE node
I20260621 06:58:21.021116 8481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:21.022538 8481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:21.023728 8481 hybrid_clock.cc:648] HybridClock initialized: now 1782025101023687 us; error 67 us; skew 500 ppm
I20260621 06:58:21.024025 8481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:21.026088 8481 webserver.cc:460] Webserver started at http://127.2.245.254:42439/ using document root <none> and password file <none>
I20260621 06:58:21.026760 8481 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:21.026847 8481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:21.027199 8481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:21.029903 8481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "c76226f948d44c7cbece49cf0f6037ac"
format_stamp: "Formatted at 2026-06-21 06:58:21 on dist-test-slave-9zdv"
server_key: "fbbf6eaae6cd405e124439af1f01b919"
server_key_iv: "745bf52c771e8123cf727cf4c1c6fbfa"
server_key_version: "encryptionkey@0"
I20260621 06:58:21.030581 8481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "c76226f948d44c7cbece49cf0f6037ac"
format_stamp: "Formatted at 2026-06-21 06:58:21 on dist-test-slave-9zdv"
server_key: "fbbf6eaae6cd405e124439af1f01b919"
server_key_iv: "745bf52c771e8123cf727cf4c1c6fbfa"
server_key_version: "encryptionkey@0"
I20260621 06:58:21.035449 8481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:58:21.038751 8495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:21.040222 8481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:21.040441 8481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "c76226f948d44c7cbece49cf0f6037ac"
format_stamp: "Formatted at 2026-06-21 06:58:21 on dist-test-slave-9zdv"
server_key: "fbbf6eaae6cd405e124439af1f01b919"
server_key_iv: "745bf52c771e8123cf727cf4c1c6fbfa"
server_key_version: "encryptionkey@0"
I20260621 06:58:21.040665 8481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:21.062016 8481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:21.062803 8481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:21.063113 8481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:21.081408 8481 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:39331
I20260621 06:58:21.081415 8546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:39331 every 8 connection(s)
I20260621 06:58:21.082727 8481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:21.086297 8547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:21.092156 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8481
I20260621 06:58:21.092346 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:21.093000 3031 external_mini_cluster.cc:1428] Setting key d1954480cce76a74386e1385352b9333
I20260621 06:58:21.097287 8547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac: Bootstrap starting.
I20260621 06:58:21.101401 8547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:21.102607 8547 log.cc:824] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:21.106015 8547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac: No bootstrap required, opened a new log
I20260621 06:58:21.108695 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:21.095063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.2.245.254:39331)
0621 06:58:21.096125 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:21.096138 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:21.096186 (+ 48us) server_negotiation.cc:407] Connection header received
0621 06:58:21.098539 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:21.098575 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:21.099242 (+ 667us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:21.099696 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:21.100243 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.102271 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.104046 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.105056 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.105261 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:21.106310 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:21.106318 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:21.106331 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:58:21.106909 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:21.107047 (+ 138us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:58:21.107069 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:21.107200 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:21.107391 (+ 191us) server_negotiation.cc:299] Negotiation successful
0621 06:58:21.107704 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":330,"threads_started":1}
I20260621 06:58:21.112438 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 06:58:21.114048 8547 raft_consensus.cc:348] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } }
I20260621 06:58:21.114460 8547 raft_consensus.cc:374] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:21.114622 8547 raft_consensus.cc:729] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76226f948d44c7cbece49cf0f6037ac, State: Initialized, Role: FOLLOWER
I20260621 06:58:21.115267 8547 consensus_queue.cc:260] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } }
I20260621 06:58:21.115516 8547 raft_consensus.cc:388] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:21.115665 8547 raft_consensus.cc:482] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:21.115844 8547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:21.117879 8547 raft_consensus.cc:504] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } }
I20260621 06:58:21.118360 8547 leader_election.cc:302] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c76226f948d44c7cbece49cf0f6037ac; no voters:
I20260621 06:58:21.118929 8547 leader_election.cc:288] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:21.119143 8552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:21.120422 8552 raft_consensus.cc:686] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [term 1 LEADER]: Becoming Leader. State: Replica: c76226f948d44c7cbece49cf0f6037ac, State: Running, Role: LEADER
I20260621 06:58:21.120929 8552 consensus_queue.cc:237] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } }
I20260621 06:58:21.121192 8547 sys_catalog.cc:564] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:21.123667 8553 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c76226f948d44c7cbece49cf0f6037ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } } }
I20260621 06:58:21.123998 8553 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:21.123781 8554 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76226f948d44c7cbece49cf0f6037ac. Latest consensus state: current_term: 1 leader_uuid: "c76226f948d44c7cbece49cf0f6037ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76226f948d44c7cbece49cf0f6037ac" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 39331 } } }
I20260621 06:58:21.124296 8554 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:21.128042 8559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:21.132946 8559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:21.141624 8559 catalog_manager.cc:1347] Generated new cluster ID: 9860771614b74f1fa91cfcf852bf450e
I20260621 06:58:21.141757 8559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:21.151055 8559 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:21.152993 8559 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:21.162766 8559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c76226f948d44c7cbece49cf0f6037ac: Generated new TSK 0
I20260621 06:58:21.163657 8559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:58:21.228062 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:21.177198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.2.245.254:39331)
0621 06:58:21.177397 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:21.177404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:21.177695 (+ 291us) server_negotiation.cc:407] Connection header received
0621 06:58:21.177886 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:21.177892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:21.177994 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:21.178166 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0621 06:58:21.178594 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.179767 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.181068 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.181748 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.181876 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:21.182317 (+ 441us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 06:58:21.227166 (+ 44849us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0621 06:58:21.227643 (+ 477us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":70}
W20260621 06:58:21.228085 8576 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:21.177004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:39331 (local address 127.0.0.1:48888)
0621 06:58:21.177579 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:21.177596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:21.177722 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:21.178171 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:21.178174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:21.178182 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0621 06:58:21.178442 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:21.178450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.179906 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.179910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:21.180928 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:21.180939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:21.181905 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:21.181909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:21.182032 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:21.182062 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 06:58:21.227503 (+ 45441us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0621 06:58:21.227798 (+ 295us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.245.254:39331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":303,"threads_started":1}
W20260621 06:58:21.228205 8550 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260621 06:58:21.229779 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8481
I20260621 06:58:21.230870 8542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:21.327476 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8481
2026-06-21T06:58:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (725 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (405 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (402 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:22 dist-test-slave-9zdv krb5kdc[8591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:22 dist-test-slave-9zdv krb5kdc[8591](info): set up 2 sockets
Jun 21 06:58:22 dist-test-slave-9zdv krb5kdc[8591](info): commencing operation
krb5kdc: starting...
W20260621 06:58:23.211722 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.008s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:23 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:23.419966 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:37751
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:23.740162 8604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:23.740442 8604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:23.740620 8604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:23.747692 8604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:23.747795 8604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:23.747886 8604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:23.747972 8604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:23.748068 8604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:23.757639 8604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:23.758970 8604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:23.760253 8604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:23.771935 8612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:23.772051 8609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:23.772030 8610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:23.772957 8604 server_base.cc:1034] running on GCE node
I20260621 06:58:23.773799 8604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:23.775254 8604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:23.776474 8604 hybrid_clock.cc:648] HybridClock initialized: now 1782025103776423 us; error 29 us; skew 500 ppm
I20260621 06:58:23.776834 8604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:23 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025103, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:23.779973 8604 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:23.781445 8604 webserver.cc:460] Webserver started at http://127.2.245.254:39305/ using document root <none> and password file <none>
I20260621 06:58:23.782195 8604 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:23.782322 8604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:23.782642 8604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:23.785575 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "9b8ad8f1d6c0447586977e487e2cbc00"
format_stamp: "Formatted at 2026-06-21 06:58:23 on dist-test-slave-9zdv"
server_key: "c1bc34d67e830d872290109c9621fa9c"
server_key_iv: "cf13220e43bbe2d6605212e3b9e9929b"
server_key_version: "encryptionkey@0"
I20260621 06:58:23.786358 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "9b8ad8f1d6c0447586977e487e2cbc00"
format_stamp: "Formatted at 2026-06-21 06:58:23 on dist-test-slave-9zdv"
server_key: "c1bc34d67e830d872290109c9621fa9c"
server_key_iv: "cf13220e43bbe2d6605212e3b9e9929b"
server_key_version: "encryptionkey@0"
I20260621 06:58:23.791384 8604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 06:58:23.794714 8619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:23.796104 8604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:23.796288 8604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "9b8ad8f1d6c0447586977e487e2cbc00"
format_stamp: "Formatted at 2026-06-21 06:58:23 on dist-test-slave-9zdv"
server_key: "c1bc34d67e830d872290109c9621fa9c"
server_key_iv: "cf13220e43bbe2d6605212e3b9e9929b"
server_key_version: "encryptionkey@0"
I20260621 06:58:23.796482 8604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:23.819326 8604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:23.823629 8604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:23.823917 8604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:23.842592 8604 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:37751
I20260621 06:58:23.842622 8670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:37751 every 8 connection(s)
I20260621 06:58:23.843935 8604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:23.847718 8671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:23.849301 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8604
I20260621 06:58:23.849515 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:23.850416 3031 external_mini_cluster.cc:1428] Setting key eb961efc54a927ad08ba3ab6bc0bd0b6
I20260621 06:58:23.862377 8671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00: Bootstrap starting.
Jun 21 06:58:23 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:23.866142 8671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:23.867408 8671 log.cc:824] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:23.870599 8671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00: No bootstrap required, opened a new log
I20260621 06:58:23.874408 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:23.852522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.2.245.254:37751)
0621 06:58:23.853427 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:23.853439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:23.853481 (+ 42us) server_negotiation.cc:407] Connection header received
0621 06:58:23.855177 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:23.855200 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:23.855603 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:23.855912 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:23.857573 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:23.858846 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:23.860709 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:23.861340 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:23.861461 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:23.866081 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:23.866089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:23.866104 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:23.866143 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:23.869930 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:23.870663 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:23.870671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:23.870679 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:23.870874 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:23.871280 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:23.871285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:23.871287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:23.871962 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:23.872236 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:23.872573 (+ 337us) server_negotiation.cc:299] Negotiation successful
0621 06:58:23.873041 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":475,"threads_started":1}
I20260621 06:58:23.880136 8671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } }
I20260621 06:58:23.880450 8671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:23.880520 8671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8ad8f1d6c0447586977e487e2cbc00, State: Initialized, Role: FOLLOWER
I20260621 06:58:23.880961 8671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } }
I20260621 06:58:23.881142 8671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:23.881234 8671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:23.881346 8671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:23.883671 8671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } }
I20260621 06:58:23.884075 8671 leader_election.cc:302] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b8ad8f1d6c0447586977e487e2cbc00; no voters:
I20260621 06:58:23.884567 8671 leader_election.cc:288] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:23.884692 8676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:23.885350 8676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8ad8f1d6c0447586977e487e2cbc00, State: Running, Role: LEADER
I20260621 06:58:23.885936 8676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } }
I20260621 06:58:23.886674 8671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:23.889793 8678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b8ad8f1d6c0447586977e487e2cbc00. Latest consensus state: current_term: 1 leader_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } } }
I20260621 06:58:23.890251 8678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:23.891181 8684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:23.891042 8677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8ad8f1d6c0447586977e487e2cbc00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37751 } } }
I20260621 06:58:23.891518 8677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:23.898443 8684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:23.907582 8684 catalog_manager.cc:1347] Generated new cluster ID: 945adb4681c2404da1bb73cf15010f07
I20260621 06:58:23.907712 8684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:23.921058 8684 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:23.922348 8684 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:23.932791 8684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b8ad8f1d6c0447586977e487e2cbc00: Generated new TSK 0
I20260621 06:58:23.933615 8684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:23.990487 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:24.332828 8697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:24.333112 8697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:24.333214 8697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:24.340369 8697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:24.340489 8697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:24.340600 8697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:24.350199 8697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:24.351413 8697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:24.352738 8697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:24.365751 8702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:24.365708 8705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:24.365788 8703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:24.367012 8697 server_base.cc:1034] running on GCE node
I20260621 06:58:24.367668 8697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:24.368572 8697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:24.369820 8697 hybrid_clock.cc:648] HybridClock initialized: now 1782025104369782 us; error 53 us; skew 500 ppm
I20260621 06:58:24.370152 8697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:24 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025104, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:24.376963 8697 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:24.378489 8697 webserver.cc:460] Webserver started at http://127.2.245.193:44001/ using document root <none> and password file <none>
I20260621 06:58:24.379222 8697 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:24.379326 8697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:24.379590 8697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:24.382412 8697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "2bfee529801d4d56ab6bd7b4795f53d4"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "0230caff03bf02ab9bc5d2367c57c606"
server_key_iv: "38d18ccd3c547049350a2de4db758d5b"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.383265 8697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "2bfee529801d4d56ab6bd7b4795f53d4"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "0230caff03bf02ab9bc5d2367c57c606"
server_key_iv: "38d18ccd3c547049350a2de4db758d5b"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.388764 8697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:58:24.392241 8712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.393783 8697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:24.393923 8697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "2bfee529801d4d56ab6bd7b4795f53d4"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "0230caff03bf02ab9bc5d2367c57c606"
server_key_iv: "38d18ccd3c547049350a2de4db758d5b"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.394112 8697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:24.410384 8697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:24.414752 8697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:24.415068 8697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:24.416002 8697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:24.417270 8697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:24.417342 8697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.417428 8697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:24.417482 8697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.450039 8697 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:37397
I20260621 06:58:24.450081 8824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:37397 every 8 connection(s)
I20260621 06:58:24.451418 8697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:24.454229 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8697
I20260621 06:58:24.454478 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:24.455256 3031 external_mini_cluster.cc:1428] Setting key 281ae0d529952881b1eff81c567dec2c
Jun 21 06:58:24 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025104, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:24.474525 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:24.454081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:44049 (local address 127.2.245.254:37751)
0621 06:58:24.454596 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:24.454602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:24.455708 (+ 1106us) server_negotiation.cc:407] Connection header received
0621 06:58:24.457981 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:24.457985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:24.458048 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:24.458143 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:24.460440 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.461167 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.463758 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.464204 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.464346 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:24.468484 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:24.468492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:24.468500 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:24.468546 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:24.471439 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:24.472208 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:24.472215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:24.472223 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:24.472335 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:24.472763 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:24.472768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:24.472771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:24.473035 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:24.473190 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:24.473919 (+ 729us) server_negotiation.cc:299] Negotiation successful
0621 06:58:24.474138 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":243,"threads_started":1}
I20260621 06:58:24.475541 8828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:24.454605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37751 (local address 127.2.245.193:44049)
0621 06:58:24.455516 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:24.455573 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:24.457631 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:24.458439 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:24.458451 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:24.459328 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:24.460172 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:24.460201 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.461335 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.461344 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:24.463585 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:24.463604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.464470 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.464479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:24.464745 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:24.465473 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:24.465507 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:24.468232 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:24.471602 (+ 3370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:24.471611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:24.471628 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:24.472058 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:24.472445 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:24.472450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:24.472454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:24.472639 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:24.473163 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:24.473171 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:24.473510 (+ 339us) client_negotiation.cc:769] Sending connection context
0621 06:58:24.473811 (+ 301us) client_negotiation.cc:240] Negotiation successful
0621 06:58:24.474338 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":363,"threads_started":1}
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:24.477171 8825 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37751
I20260621 06:58:24.477445 8825 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:24.478279 8825 heartbeater.cc:511] Master 127.2.245.254:37751 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:24.480515 8636 ts_manager.cc:194] Registered new tserver with Master: 2bfee529801d4d56ab6bd7b4795f53d4 (127.2.245.193:37397)
I20260621 06:58:24.482005 8636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:44049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:24.503657 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:24.818997 8831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:24.819278 8831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:24.819370 8831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:24.826375 8831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:24.826496 8831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:24.826601 8831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:24.836062 8831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:24.837301 8831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:24.838521 8831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:24.852241 8836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:24.852268 8837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:24.852826 8839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:24.853032 8831 server_base.cc:1034] running on GCE node
I20260621 06:58:24.853816 8831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:24.854898 8831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:24.856158 8831 hybrid_clock.cc:648] HybridClock initialized: now 1782025104856106 us; error 44 us; skew 500 ppm
I20260621 06:58:24.856515 8831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:24 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025104, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:24.860005 8831 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:24.861490 8831 webserver.cc:460] Webserver started at http://127.2.245.194:45807/ using document root <none> and password file <none>
I20260621 06:58:24.862178 8831 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:24.862318 8831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:24.862629 8831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:24.865476 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "164a1bfa5f894b569d18e5f8c6eaec67"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "4bef846bdb172631c1602a6a6deb8249"
server_key_iv: "714057fccd0715323a48e3222071622e"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.866209 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "164a1bfa5f894b569d18e5f8c6eaec67"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "4bef846bdb172631c1602a6a6deb8249"
server_key_iv: "714057fccd0715323a48e3222071622e"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.871244 8831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260621 06:58:24.874701 8846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.876065 8831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:24.876286 8831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "164a1bfa5f894b569d18e5f8c6eaec67"
format_stamp: "Formatted at 2026-06-21 06:58:24 on dist-test-slave-9zdv"
server_key: "4bef846bdb172631c1602a6a6deb8249"
server_key_iv: "714057fccd0715323a48e3222071622e"
server_key_version: "encryptionkey@0"
I20260621 06:58:24.876487 8831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:24.896922 8831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:24.901383 8831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:24.901685 8831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:24.902686 8831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:24.904053 8831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:24.904158 8831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.904299 8831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:24.904367 8831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:24.938212 8831 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:37279
I20260621 06:58:24.938303 8958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:37279 every 8 connection(s)
I20260621 06:58:24.939597 8831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:24.943225 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8831
I20260621 06:58:24.943398 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:24.943956 3031 external_mini_cluster.cc:1428] Setting key 61c5ae41f13d0c1beb4a004047c1a863
Jun 21 06:58:24 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025104, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:24.962529 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:24.942102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:35665 (local address 127.2.245.254:37751)
0621 06:58:24.942272 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:24.942276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:24.943739 (+ 1463us) server_negotiation.cc:407] Connection header received
0621 06:58:24.945890 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:24.945895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:24.945978 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:24.946088 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:24.948158 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.948874 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.951413 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.951860 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.951961 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:24.956317 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:24.956321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:24.956324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:24.956356 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:24.958900 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:24.959922 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:24.959925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:24.959928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:24.960048 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:24.960569 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:24.960573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:24.960575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:24.960832 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:24.960951 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:24.961933 (+ 982us) server_negotiation.cc:299] Negotiation successful
0621 06:58:24.962128 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 06:58:24.964071 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:24.942542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37751 (local address 127.2.245.194:35665)
0621 06:58:24.943567 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:24.943621 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:24.945597 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:24.946337 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:24.946350 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:24.947121 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:24.947953 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:24.947982 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.949041 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.949050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:24.951250 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:24.951269 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:24.952092 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:24.952101 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:24.952426 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:24.953296 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:24.953322 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:24.956068 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:24.959098 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:24.959110 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:24.959131 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:24.959799 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:24.960165 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:24.960172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:24.960179 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:24.960429 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:24.960978 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:24.960989 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:24.961479 (+ 490us) client_negotiation.cc:769] Sending connection context
0621 06:58:24.961801 (+ 322us) client_negotiation.cc:240] Negotiation successful
0621 06:58:24.962424 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":491,"threads_started":1}
I20260621 06:58:24.965659 8959 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37751
I20260621 06:58:24.966003 8959 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:58:24.966719 8959 heartbeater.cc:511] Master 127.2.245.254:37751 requested a full tablet report, sending...
I20260621 06:58:24.968281 8636 ts_manager.cc:194] Registered new tserver with Master: 164a1bfa5f894b569d18e5f8c6eaec67 (127.2.245.194:37279)
I20260621 06:58:24.969156 8636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:35665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:24.994156 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:25.308516 8964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:25.308849 8964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:25.309031 8964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:25.316370 8964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:25.316527 8964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:25.316715 8964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:25.326606 8964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:25.327955 8964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:25.329303 8964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:25.343308 8969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:25.343261 8972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:25.343222 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:25.344038 8964 server_base.cc:1034] running on GCE node
I20260621 06:58:25.344691 8964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:25.345674 8964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:25.346864 8964 hybrid_clock.cc:648] HybridClock initialized: now 1782025105346802 us; error 56 us; skew 500 ppm
I20260621 06:58:25.347210 8964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.350397 8964 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:25.351893 8964 webserver.cc:460] Webserver started at http://127.2.245.195:37965/ using document root <none> and password file <none>
I20260621 06:58:25.352598 8964 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:25.352697 8964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:25.352944 8964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:25.355847 8964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "e64becd45a8e4da3b1e3dbecad24fa1f"
format_stamp: "Formatted at 2026-06-21 06:58:25 on dist-test-slave-9zdv"
server_key: "fdf3f9af7c980bbf2d766a467e79be79"
server_key_iv: "9ebafc4f35fcc22c2224d21df7c7ae67"
server_key_version: "encryptionkey@0"
I20260621 06:58:25.356564 8964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "e64becd45a8e4da3b1e3dbecad24fa1f"
format_stamp: "Formatted at 2026-06-21 06:58:25 on dist-test-slave-9zdv"
server_key: "fdf3f9af7c980bbf2d766a467e79be79"
server_key_iv: "9ebafc4f35fcc22c2224d21df7c7ae67"
server_key_version: "encryptionkey@0"
I20260621 06:58:25.361629 8964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 06:58:25.365052 8979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:25.366358 8964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:25.366484 8964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "e64becd45a8e4da3b1e3dbecad24fa1f"
format_stamp: "Formatted at 2026-06-21 06:58:25 on dist-test-slave-9zdv"
server_key: "fdf3f9af7c980bbf2d766a467e79be79"
server_key_iv: "9ebafc4f35fcc22c2224d21df7c7ae67"
server_key_version: "encryptionkey@0"
I20260621 06:58:25.366652 8964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:25.400367 8964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:25.406606 8964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:25.407048 8964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:25.408334 8964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:25.410089 8964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:25.410197 8964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:25.410324 8964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:25.410394 8964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:25.445600 8964 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:46825
I20260621 06:58:25.445647 9091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:46825 every 8 connection(s)
I20260621 06:58:25.446870 8964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:25.454177 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 8964
I20260621 06:58:25.454378 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:25.454926 3031 external_mini_cluster.cc:1428] Setting key d7d9d38556b22195075c406c54539453
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:25.466897 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.449495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:49139 (local address 127.2.245.254:37751)
0621 06:58:25.449717 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.449723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.451020 (+ 1297us) server_negotiation.cc:407] Connection header received
0621 06:58:25.452801 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.452806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.452889 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.453021 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.454669 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.455739 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.457349 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.457914 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.458070 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.461431 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.461436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.461440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.461470 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.463911 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.464624 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.464632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.464635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.464766 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.465208 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.465214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.465217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.465485 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.465641 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.466310 (+ 669us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.466525 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 06:58:25.467913 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.450001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37751 (local address 127.2.245.195:49139)
0621 06:58:25.450820 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:25.450865 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:25.452519 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:25.453223 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:25.453232 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:25.453826 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:25.454434 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:25.454496 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.455920 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.455924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:25.457162 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:25.457173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.458138 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.458146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:25.458411 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.459074 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:25.459096 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:25.461204 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:25.464060 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:25.464069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:25.464086 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:25.464492 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:25.464880 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:25.464884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:25.464888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:25.465092 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:25.465616 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:25.465623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:25.465953 (+ 330us) client_negotiation.cc:769] Sending connection context
0621 06:58:25.466219 (+ 266us) client_negotiation.cc:240] Negotiation successful
0621 06:58:25.466744 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":354,"threads_started":1}
I20260621 06:58:25.469333 9092 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37751
I20260621 06:58:25.469684 9092 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:25.470571 9092 heartbeater.cc:511] Master 127.2.245.254:37751 requested a full tablet report, sending...
I20260621 06:58:25.471942 8635 ts_manager.cc:194] Registered new tserver with Master: e64becd45a8e4da3b1e3dbecad24fa1f (127.2.245.195:46825)
I20260621 06:58:25.472820 8635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:49139
I20260621 06:58:25.482440 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:25.485381 8825 heartbeater.cc:503] Master 127.2.245.254:37751 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/test-admin.keytab.
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.509778 3031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260621 06:58:25.514667 9099 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:25.512342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.2.245.193:37397)
0621 06:58:25.512957 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.512963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.512992 (+ 29us) server_negotiation.cc:407] Connection header received
0621 06:58:25.513055 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.513061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.513263 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.513391 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.514221 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35626: BlockingRecv error: recv got EOF from 127.0.0.1:35626 (error 108)
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":296,"threads_started":1}
W20260621 06:58:25.514698 3031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc' of type FILE
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.516510 3031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.517827 3031 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:25.532413 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.519380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.2.245.193:37397)
0621 06:58:25.519642 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.519648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.519807 (+ 159us) server_negotiation.cc:407] Connection header received
0621 06:58:25.519978 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.519982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.520042 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.520125 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.521074 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.522016 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.523233 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.523769 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.523861 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.527257 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.527271 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.527285 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.527321 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.529828 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.530358 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.530364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.530366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.530490 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.530803 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.530806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.530808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.531191 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.531396 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.531730 (+ 334us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.532003 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
W20260621 06:58:25.538303 9099 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:25.536444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.2.245.193:37397)
0621 06:58:25.536675 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.536685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.537121 (+ 436us) server_negotiation.cc:407] Connection header received
0621 06:58:25.537250 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.537254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.537334 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.537410 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.538076 (+ 666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35644: BlockingRecv error: recv got EOF from 127.0.0.1:35644 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260621 06:58:25.538527 3031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc' of type FILE
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.540071 3031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.541285 3031 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:25.554347 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.542427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.2.245.193:37397)
0621 06:58:25.542602 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.542610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.542884 (+ 274us) server_negotiation.cc:407] Connection header received
0621 06:58:25.543083 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.543087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.543140 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.543217 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.544117 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.544758 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.545875 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.546286 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.546374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.549671 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.549676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.549679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.549720 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.552228 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.552776 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.552780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.552782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.552890 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.553193 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.553195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.553197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.553443 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.553662 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.553837 (+ 175us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.554006 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260621 06:58:25.558317 9099 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:25.556644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.2.245.193:37397)
0621 06:58:25.556831 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.556837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.557090 (+ 253us) server_negotiation.cc:407] Connection header received
0621 06:58:25.557257 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.557262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.557339 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.557439 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.558107 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35656: BlockingRecv error: recv got EOF from 127.0.0.1:35656 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260621 06:58:25.558547 3031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782025057166075-3031-0/krb5kdc/krb5cc' of type FILE
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.559990 3031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:25.561182 3031 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:25.574678 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.562474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.2.245.193:37397)
0621 06:58:25.562676 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.562682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.562858 (+ 176us) server_negotiation.cc:407] Connection header received
0621 06:58:25.563020 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.563025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.563080 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.563157 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.563999 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.564662 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.565955 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.566364 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.566453 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.569814 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.569818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.569821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.569850 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.572504 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.573071 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.573074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.573076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.573196 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.573486 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.573488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.573490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.573739 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.573971 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.574149 (+ 178us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.574342 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 06:58:25.576289 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8697
I20260621 06:58:25.587282 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.577007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.2.245.193:37397)
0621 06:58:25.577163 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.577167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.577441 (+ 274us) server_negotiation.cc:407] Connection header received
0621 06:58:25.577552 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.577574 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.577623 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.577693 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.578490 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.579317 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.580534 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.580936 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.581024 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.582816 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.582821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.582824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.582847 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.585135 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.585649 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.585653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.585655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.585761 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.586080 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.586083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.586085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.586292 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.586468 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.586853 (+ 385us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.587064 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 06:58:25.587819 8820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:25.669183 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8697
I20260621 06:58:25.685204 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8831
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:58:25.702569 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.686162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.2.245.194:37279)
0621 06:58:25.686682 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.686689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.686713 (+ 24us) server_negotiation.cc:407] Connection header received
0621 06:58:25.686799 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.686809 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.687135 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.687325 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.688853 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.690598 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.692601 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.693203 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.693351 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.697243 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.697254 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.697267 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.697297 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.700080 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.700658 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.700664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.700666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.700780 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.701180 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.701183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.701185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.701516 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.701732 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.701998 (+ 266us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.702248 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":262,"threads_started":1}
I20260621 06:58:25.703845 8954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:25.784135 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8831
I20260621 06:58:25.800271 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8964
Jun 21 06:58:25 dist-test-slave-9zdv krb5kdc[8591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:58:25.816321 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:25.801385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.2.245.195:46825)
0621 06:58:25.802136 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:25.802143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:25.802172 (+ 29us) server_negotiation.cc:407] Connection header received
0621 06:58:25.802230 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:25.802236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:25.802443 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:25.802581 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:25.803599 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.804539 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.805699 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:25.806212 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:25.806311 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:25.810199 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:25.810211 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:25.810226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:25.810256 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:25.813406 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.814183 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.814192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.814195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.814335 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:25.814853 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:25.814859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:25.814862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:25.815346 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:25.815650 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:25.815863 (+ 213us) server_negotiation.cc:299] Negotiation successful
0621 06:58:25.816051 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":422,"threads_started":1}
I20260621 06:58:25.817648 9087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:25.907478 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8964
I20260621 06:58:25.923166 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 8604
I20260621 06:58:25.924383 8666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:26.027809 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 8604
2026-06-21T06:58:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3898 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:26 dist-test-slave-9zdv krb5kdc[9119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:26 dist-test-slave-9zdv krb5kdc[9119](info): set up 2 sockets
Jun 21 06:58:26 dist-test-slave-9zdv krb5kdc[9119](info): commencing operation
krb5kdc: starting...
W20260621 06:58:27.103826 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:27 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab.
Entry for principal oryx/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab.
I20260621 06:58:27.304203 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38303
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab
--principal=oryx/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:27.619994 9132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:27.620251 9132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:27.620332 9132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:27.627173 9132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:27.627276 9132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:27.627321 9132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:27.627375 9132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:27.627416 9132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:27.636678 9132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab
--principal=oryx/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:27.637892 9132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:27.639163 9132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:27.650450 9137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:27.650436 9138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:27.650607 9140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:27.651285 9132 server_base.cc:1034] running on GCE node
I20260621 06:58:27.652021 9132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:27.653332 9132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:27.654570 9132 hybrid_clock.cc:648] HybridClock initialized: now 1782025107654534 us; error 58 us; skew 500 ppm
I20260621 06:58:27.654975 9132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:27 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:27.657969 9132 init.cc:377] Logged in from keytab as oryx/127.2.245.254@KRBTEST.COM (short username oryx)
I20260621 06:58:27.659518 9132 webserver.cc:460] Webserver started at http://127.2.245.254:41291/ using document root <none> and password file <none>
I20260621 06:58:27.660218 9132 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:27.660356 9132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:27.660679 9132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:27.663709 9132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2"
format_stamp: "Formatted at 2026-06-21 06:58:27 on dist-test-slave-9zdv"
server_key: "1164f4dfb6cbd0fbdfe67401534d3b2b"
server_key_iv: "a79045659dfb6e6a24ea22c26187b42d"
server_key_version: "encryptionkey@0"
I20260621 06:58:27.664433 9132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2"
format_stamp: "Formatted at 2026-06-21 06:58:27 on dist-test-slave-9zdv"
server_key: "1164f4dfb6cbd0fbdfe67401534d3b2b"
server_key_iv: "a79045659dfb6e6a24ea22c26187b42d"
server_key_version: "encryptionkey@0"
I20260621 06:58:27.669490 9132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:27.672961 9147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:27.674310 9132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:27.674481 9132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2"
format_stamp: "Formatted at 2026-06-21 06:58:27 on dist-test-slave-9zdv"
server_key: "1164f4dfb6cbd0fbdfe67401534d3b2b"
server_key_iv: "a79045659dfb6e6a24ea22c26187b42d"
server_key_version: "encryptionkey@0"
I20260621 06:58:27.675243 9132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:27.698839 9132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:27.702833 9132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:27.703145 9132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:27.720619 9132 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:38303
I20260621 06:58:27.720701 9198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:38303 every 8 connection(s)
I20260621 06:58:27.721956 9132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:27.722422 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9132
I20260621 06:58:27.722625 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:27.723464 3031 external_mini_cluster.cc:1428] Setting key 3b4edef59ce1fad1f5cc5e2b79671101
I20260621 06:58:27.726135 9199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 06:58:27 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:27.739101 9199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Bootstrap starting.
I20260621 06:58:27.743199 9199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:27.744235 9199 log.cc:824] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:27.746632 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:27.725592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.2.245.254:38303)
0621 06:58:27.726341 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:27.726353 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:27.726391 (+ 38us) server_negotiation.cc:407] Connection header received
0621 06:58:27.728036 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:27.728057 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:27.728408 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:27.728753 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:27.730471 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:27.731846 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:27.733568 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:27.734183 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:27.734306 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:27.738870 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:27.738882 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:27.738900 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:27.738989 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:27.742694 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:27.743385 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:27.743402 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:27.743413 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:27.743579 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:27.743961 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:27.743967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:27.743970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:27.744549 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:27.744866 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:27.745310 (+ 444us) server_negotiation.cc:299] Negotiation successful
0621 06:58:27.745721 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":343,"threads_started":1}
I20260621 06:58:27.747201 9199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: No bootstrap required, opened a new log
I20260621 06:58:27.754562 9199 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } }
I20260621 06:58:27.755079 9199 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:27.755245 9199 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb53aeb7e9f94cdaaff92e1a3acbfec2, State: Initialized, Role: FOLLOWER
I20260621 06:58:27.755894 9199 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } }
I20260621 06:58:27.756165 9199 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:27.756300 9199 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:27.756469 9199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:27.758467 9199 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } }
I20260621 06:58:27.759023 9199 leader_election.cc:302] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb53aeb7e9f94cdaaff92e1a3acbfec2; no voters:
I20260621 06:58:27.759723 9199 leader_election.cc:288] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:27.759850 9204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:27.760553 9204 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 LEADER]: Becoming Leader. State: Replica: cb53aeb7e9f94cdaaff92e1a3acbfec2, State: Running, Role: LEADER
I20260621 06:58:27.761097 9204 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } }
I20260621 06:58:27.761718 9199 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:27.763875 9206 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb53aeb7e9f94cdaaff92e1a3acbfec2. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } }
I20260621 06:58:27.763967 9205 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } }
I20260621 06:58:27.764343 9206 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:27.764339 9205 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:27.767462 9212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:27.771622 9212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:27.780006 9212 catalog_manager.cc:1347] Generated new cluster ID: e862c6e13ebc41fdb7b68be0dee74ebf
I20260621 06:58:27.780156 9212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:27.793697 9212 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:27.795547 9212 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:27.813792 9212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Generated new TSK 0
I20260621 06:58:27.814879 9212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
I20260621 06:58:27.873083 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:28.195283 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:28.195588 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:28.195761 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:28.202559 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:28.202701 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:28.202862 9225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:28.212479 9225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:28.213778 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:28.215201 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:28.228394 9230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:28.228431 9231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:28.228410 9233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:28.229461 9225 server_base.cc:1034] running on GCE node
I20260621 06:58:28.230075 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:28.231058 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:28.232290 9225 hybrid_clock.cc:648] HybridClock initialized: now 1782025108232245 us; error 40 us; skew 500 ppm
I20260621 06:58:28.232646 9225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:28 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025108, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:28.236400 9225 init.cc:377] Logged in from keytab as oryx/127.2.245.193@KRBTEST.COM (short username oryx)
I20260621 06:58:28.237856 9225 webserver.cc:460] Webserver started at http://127.2.245.193:35795/ using document root <none> and password file <none>
I20260621 06:58:28.238513 9225 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:28.238610 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:28.238862 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:28.241747 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "c782c8644c7440eb9a3befd68a4fc500"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "63428ed120e457fe8e3a8b794b07fa37"
server_key_iv: "20aa97fde14168b5c3f6316e2d708e7c"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.242415 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "c782c8644c7440eb9a3befd68a4fc500"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "63428ed120e457fe8e3a8b794b07fa37"
server_key_iv: "20aa97fde14168b5c3f6316e2d708e7c"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.247864 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260621 06:58:28.251271 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.252555 9225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:28.252691 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "c782c8644c7440eb9a3befd68a4fc500"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "63428ed120e457fe8e3a8b794b07fa37"
server_key_iv: "20aa97fde14168b5c3f6316e2d708e7c"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.252876 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:28.286635 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:28.291067 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:28.291318 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:28.292199 9225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:28.293581 9225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:28.293655 9225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.293754 9225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:28.293792 9225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.326783 9225 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:43637
I20260621 06:58:28.326874 9352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:43637 every 8 connection(s)
I20260621 06:58:28.328150 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:28.333773 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9225
I20260621 06:58:28.334005 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:28.334853 3031 external_mini_cluster.cc:1428] Setting key 4968a4fb0ace7dd4a410a153612dd01d
Jun 21 06:58:28 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025108, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:28.349985 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:28.330657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:55585 (local address 127.2.245.254:38303)
0621 06:58:28.331116 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:28.331122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:28.332134 (+ 1012us) server_negotiation.cc:407] Connection header received
0621 06:58:28.334202 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:28.334220 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:28.334289 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:28.334390 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:28.336402 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.337185 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.339801 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.340174 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.340296 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:28.344381 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:28.344386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:28.344389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:28.344418 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:28.347033 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:28.347724 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:28.347731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:28.347735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:28.347853 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:28.348275 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:28.348280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:28.348283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:28.348530 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:28.348680 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:28.349405 (+ 725us) server_negotiation.cc:299] Negotiation successful
0621 06:58:28.349626 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":214,"threads_started":1}
I20260621 06:58:28.351372 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:28.331327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:55585)
0621 06:58:28.331971 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:28.332008 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:28.333851 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:28.334721 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:28.334735 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:28.335512 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:28.336238 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:28.336257 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.337367 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.337373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:28.339624 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:28.339641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.340427 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.340436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:28.340752 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:28.341502 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:28.341540 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:28.344156 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:28.347171 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:28.347180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:28.347198 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:28.347597 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:28.347962 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:28.347968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:28.347971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:28.348155 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:28.348656 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:28.348663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:28.349025 (+ 362us) client_negotiation.cc:769] Sending connection context
0621 06:58:28.349294 (+ 269us) client_negotiation.cc:240] Negotiation successful
0621 06:58:28.349865 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":250,"threads_started":1}
I20260621 06:58:28.353041 9353 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:28.353404 9353 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:28.354257 9353 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:28.356760 9164 ts_manager.cc:194] Registered new tserver with Master: c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637)
I20260621 06:58:28.358328 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:55585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
I20260621 06:58:28.382447 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:28.701116 9359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:28.701386 9359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:28.701550 9359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:28.708632 9359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:28.708742 9359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:28.708973 9359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:28.719218 9359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:28.720530 9359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:28.721824 9359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:28.735002 9367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:28.734900 9365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:28.735325 9364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:28.738294 9359 server_base.cc:1034] running on GCE node
I20260621 06:58:28.738914 9359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:28.739926 9359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:28.741135 9359 hybrid_clock.cc:648] HybridClock initialized: now 1782025108741073 us; error 69 us; skew 500 ppm
I20260621 06:58:28.741469 9359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:28 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025108, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:28.744582 9359 init.cc:377] Logged in from keytab as oryx/127.2.245.194@KRBTEST.COM (short username oryx)
I20260621 06:58:28.746021 9359 webserver.cc:460] Webserver started at http://127.2.245.194:43451/ using document root <none> and password file <none>
I20260621 06:58:28.746747 9359 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:28.746891 9359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:28.747344 9359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:28.750257 9359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "10ea24b03a784601b4d2b9c9dca562db"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "338071caf3368d1d122a74bc83041948"
server_key_iv: "4fdf150ed1ed5f87ae6b457b08b39e74"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.751112 9359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "10ea24b03a784601b4d2b9c9dca562db"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "338071caf3368d1d122a74bc83041948"
server_key_iv: "4fdf150ed1ed5f87ae6b457b08b39e74"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.756243 9359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 06:58:28.759832 9374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.761150 9359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260621 06:58:28.761341 9359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "10ea24b03a784601b4d2b9c9dca562db"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "338071caf3368d1d122a74bc83041948"
server_key_iv: "4fdf150ed1ed5f87ae6b457b08b39e74"
server_key_version: "encryptionkey@0"
I20260621 06:58:28.761554 9359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:28.773211 9359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:28.777148 9359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:28.777448 9359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:28.778374 9359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:28.779758 9359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:28.779861 9359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.780009 9359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:28.780102 9359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:28.812662 9359 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:34131
I20260621 06:58:28.812704 9486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:34131 every 8 connection(s)
I20260621 06:58:28.813920 9359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:28.822413 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9359
I20260621 06:58:28.822597 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:28.823246 3031 external_mini_cluster.cc:1428] Setting key 19aa5be0d91ca73738005e96a92e3362
Jun 21 06:58:28 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025108, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:28.837260 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:28.816497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:37451 (local address 127.2.245.254:38303)
0621 06:58:28.816700 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:28.816706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:28.817893 (+ 1187us) server_negotiation.cc:407] Connection header received
0621 06:58:28.819708 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:28.819713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:28.819781 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:28.819869 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:28.821260 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.821938 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.824024 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.824782 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.824911 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:28.831508 (+ 6597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:28.831513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:28.831516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:28.831549 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:28.834064 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:28.835035 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:28.835039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:28.835041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:28.835160 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:28.835652 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:28.835655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:28.835657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:28.835909 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:28.836015 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:28.836884 (+ 869us) server_negotiation.cc:299] Negotiation successful
0621 06:58:28.837033 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 06:58:28.838840 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:28.817053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:37451)
0621 06:58:28.817722 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:28.817769 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:28.819462 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:28.820062 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:28.820069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:28.820593 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:28.821107 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:28.821124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.822090 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.822094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:28.823842 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:28.823861 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:28.825144 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:28.825154 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:28.825436 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:28.828926 (+ 3490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:28.828949 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:28.831249 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:28.834197 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:28.834208 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:28.834234 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:28.834872 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:28.835271 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:28.835278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:28.835284 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:28.835537 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:28.836033 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:28.836044 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:28.836518 (+ 474us) client_negotiation.cc:769] Sending connection context
0621 06:58:28.836813 (+ 295us) client_negotiation.cc:240] Negotiation successful
0621 06:58:28.837377 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":253,"threads_started":1}
I20260621 06:58:28.840436 9487 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:28.840773 9487 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:28.841666 9487 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
I20260621 06:58:28.843396 9164 ts_manager.cc:194] Registered new tserver with Master: 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131)
WARNING: no policy specified for oryx/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:58:28.844589 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:37451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
I20260621 06:58:28.869735 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:29.198854 9492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:29.199214 9492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:29.199313 9492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:29.206219 9492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:29.206326 9492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:29.206459 9492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:29.216215 9492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:29.217422 9492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:29.218784 9492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:29.231868 9498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:29.231901 9500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:29.231812 9497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:29.232858 9492 server_base.cc:1034] running on GCE node
I20260621 06:58:29.233418 9492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:29.234288 9492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:29.235515 9492 hybrid_clock.cc:648] HybridClock initialized: now 1782025109235470 us; error 60 us; skew 500 ppm
I20260621 06:58:29.235839 9492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025109, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:29.239027 9492 init.cc:377] Logged in from keytab as oryx/127.2.245.195@KRBTEST.COM (short username oryx)
I20260621 06:58:29.240373 9492 webserver.cc:460] Webserver started at http://127.2.245.195:38211/ using document root <none> and password file <none>
I20260621 06:58:29.241004 9492 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:29.241097 9492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:29.241324 9492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:29.244087 9492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "593bc2a97b0944ea9d60eb9161798972"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "e02ca8f64645bad09e42ddf1a9507eae"
server_key_iv: "59ec0821f78a2aa1587c0d5c77507177"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.244756 9492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "593bc2a97b0944ea9d60eb9161798972"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "e02ca8f64645bad09e42ddf1a9507eae"
server_key_iv: "59ec0821f78a2aa1587c0d5c77507177"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.249748 9492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:58:29.253012 9507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:29.254343 9492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:29.254480 9492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "593bc2a97b0944ea9d60eb9161798972"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "e02ca8f64645bad09e42ddf1a9507eae"
server_key_iv: "59ec0821f78a2aa1587c0d5c77507177"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.254637 9492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:29.271924 9492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:29.275635 9492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:29.275859 9492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:29.276690 9492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:29.277872 9492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:29.277941 9492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:29.278079 9492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:29.278139 9492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:29.310550 9492 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:33905
I20260621 06:58:29.310588 9619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:33905 every 8 connection(s)
I20260621 06:58:29.311830 9492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:29.319960 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9492
I20260621 06:58:29.320140 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:29.320611 3031 external_mini_cluster.cc:1428] Setting key ca0682dc6c6f90fab468f7db837a5484
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025109, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:29.332350 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.314393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:58343 (local address 127.2.245.254:38303)
0621 06:58:29.314559 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.314564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.315891 (+ 1327us) server_negotiation.cc:407] Connection header received
0621 06:58:29.317650 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.317657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.317765 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.317883 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.319428 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.320342 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.322210 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.322614 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.322739 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.326022 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.326030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.326039 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.326069 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.329031 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.329952 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.329958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.329962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.330094 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.330598 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.330604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.330607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.330920 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.331139 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.331798 (+ 659us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.332024 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 06:58:29.333218 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.314923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:58343)
0621 06:58:29.315730 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.315778 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.317360 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.318138 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.318146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.318609 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.319256 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.319275 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.320502 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.320507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.322074 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.322085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.322836 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.322843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.323123 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.323729 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.323750 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.325832 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.329174 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.329183 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.329198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.329800 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.330211 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.330216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.330220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.330480 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.331107 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.331115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.331449 (+ 334us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.331698 (+ 249us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.332176 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":269,"threads_started":1}
I20260621 06:58:29.334702 9620 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:29.335067 9620 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:29.335965 9620 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
I20260621 06:58:29.337191 9164 ts_manager.cc:194] Registered new tserver with Master: 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:29.337992 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:58343
I20260621 06:58:29.348059 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:29.361200 9353 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.2.245.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.253.keytab.
Entry for principal oryx/127.2.245.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.253.keytab.
I20260621 06:58:29.394353 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:33397
--webserver_interface=127.2.245.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.253.keytab
--principal=oryx/127.2.245.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38303,127.2.245.253:33397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:29.715994 9625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:29.716334 9625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:29.716488 9625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:29.723850 9625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:29.723987 9625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:29.724095 9625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:29.724193 9625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:29.724272 9625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:29.734009 9625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38303,127.2.245.253:33397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.253.keytab
--principal=oryx/127.2.245.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:33397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.245.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
W20260621 06:58:29.734403 9625 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.245.254:38303,127.2.245.253:33397'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260621 06:58:29.735531 9625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:29.736807 9625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:29.748569 9630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:29.748514 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:29.748484 9631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:29.749219 9625 server_base.cc:1034] running on GCE node
I20260621 06:58:29.749927 9625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:29.751310 9625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:29.752565 9625 hybrid_clock.cc:648] HybridClock initialized: now 1782025109752503 us; error 34 us; skew 500 ppm
I20260621 06:58:29.752915 9625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025109, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:29.759991 9625 init.cc:377] Logged in from keytab as oryx/127.2.245.253@KRBTEST.COM (short username oryx)
I20260621 06:58:29.761474 9625 webserver.cc:460] Webserver started at http://127.2.245.253:37533/ using document root <none> and password file <none>
I20260621 06:58:29.762259 9625 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:29.762357 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:29.762733 9625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:29.765873 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/instance:
uuid: "0dc56c3e4389491d9d909f0662822aa8"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "6bff76dcb52b92831c8bb5442ce5466d"
server_key_iv: "e1bdc9d849d80c4c266308ad4a4bd024"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.766698 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal/instance:
uuid: "0dc56c3e4389491d9d909f0662822aa8"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "6bff76dcb52b92831c8bb5442ce5466d"
server_key_iv: "e1bdc9d849d80c4c266308ad4a4bd024"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.771751 9625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260621 06:58:29.775169 9640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:29.776530 9625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:29.776703 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "0dc56c3e4389491d9d909f0662822aa8"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "6bff76dcb52b92831c8bb5442ce5466d"
server_key_iv: "e1bdc9d849d80c4c266308ad4a4bd024"
server_key_version: "encryptionkey@0"
I20260621 06:58:29.776926 9625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:29.809929 9625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:29.814049 9625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:29.814292 9625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:29.840507 9625 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.253:33397
I20260621 06:58:29.840526 9691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.253:33397 every 8 connection(s)
I20260621 06:58:29.841925 9625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
I20260621 06:58:29.845541 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9625
I20260621 06:58:29.845614 9692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:29.847559 9487 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
I20260621 06:58:29.855427 9692 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025109, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.253@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:29.870476 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.847798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.2.245.253:33397)
0621 06:58:29.848739 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.848757 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.848806 (+ 49us) server_negotiation.cc:407] Connection header received
0621 06:58:29.850566 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.850587 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.851017 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.851376 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.853164 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.854417 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.856290 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.857179 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.857313 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.862174 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.862182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.862192 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.862224 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.865434 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.866351 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.866357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.866362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.866627 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.867140 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.867145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.867149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.867877 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.868156 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.868612 (+ 456us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.869040 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":433,"threads_started":1}
I20260621 06:58:29.874895 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.858483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.2.245.254:38303)
0621 06:58:29.859841 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.859848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.859868 (+ 20us) server_negotiation.cc:407] Connection header received
0621 06:58:29.859931 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.859936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.860024 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.860145 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.861183 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.862070 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.863747 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.864437 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.864570 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.868590 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.868597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.868602 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.868638 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.872397 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.873224 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.873236 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.873238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.873355 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.873676 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.873679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.873681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.873970 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.874074 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.874466 (+ 392us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.874610 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":1096,"threads_started":1}
I20260621 06:58:29.874903 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.858412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36650)
0621 06:58:29.859263 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.859325 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.859504 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.860195 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.860201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.860814 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.861036 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.861044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.862221 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.862228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.863574 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.863586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.864681 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.864690 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.864868 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.865510 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.865542 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.868411 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.872526 (+ 4115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.872535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.872539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.873099 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.873458 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.873462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.873464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.873582 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.874080 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.874085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.874215 (+ 130us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.874375 (+ 160us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.874585 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":291,"threads_started":1}
I20260621 06:58:29.876392 9692 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025109, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.253@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
I20260621 06:58:29.887928 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.877084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42116)
0621 06:58:29.877245 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.877259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.877370 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.877624 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.877627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.877962 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.878075 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.878082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.879161 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.879167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.880065 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.880076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.880763 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.880767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.880963 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.881404 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.881418 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.883272 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.885666 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.885674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.885678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.886011 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.886341 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.886347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.886350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.886464 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.886991 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.886995 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.887122 (+ 127us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.887253 (+ 131us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.887545 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260621 06:58:29.887969 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.877245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.2.245.253:33397)
0621 06:58:29.877385 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.877390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.877406 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:29.877459 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.877463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.877519 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.877612 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.878238 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.879034 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.880203 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.880632 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.880754 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.883434 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.883444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.883446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.883471 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.885543 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.886119 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.886127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.886130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.886234 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.886559 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.886566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.886570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.886837 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.887001 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.887339 (+ 338us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.887686 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":181,"server-negotiator.queue_time_us":46}
I20260621 06:58:29.892547 9692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Bootstrap starting.
I20260621 06:58:29.895696 9692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:29.896622 9692 log.cc:824] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:29.899137 9692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: No bootstrap required, opened a new log
I20260621 06:58:29.905843 9692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } }
I20260621 06:58:29.906126 9692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:29.906208 9692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc56c3e4389491d9d909f0662822aa8, State: Initialized, Role: FOLLOWER
I20260621 06:58:29.906703 9692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } }
I20260621 06:58:29.907497 9699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } }
I20260621 06:58:29.907855 9699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:29.908521 9692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: configured and running, proceeding with master startup.
W20260621 06:58:29.916738 9710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 06:58:29.916916 9710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 06:58:29.918776 9625 master_runner.cc:428] Detected that this master 0dc56c3e4389491d9d909f0662822aa8 is joining an existing cluster
I20260621 06:58:29.918885 9625 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.245.253:33397
I20260621 06:58:29.941738 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.923187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.2.245.254:38303)
0621 06:58:29.923517 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.923522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.923986 (+ 464us) server_negotiation.cc:407] Connection header received
0621 06:58:29.924249 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.924255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.924377 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.924448 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.925936 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.927217 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.930372 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.931124 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.931240 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.934344 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.934353 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.934358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.934402 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.937574 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.939115 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.939124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.939131 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.939291 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.939785 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.939793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.939800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.940214 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.940340 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.941030 (+ 690us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.941300 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260621 06:58:29.943251 9716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.922551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36654)
0621 06:58:29.923892 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.923909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.924081 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.924660 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.924665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.925349 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.925767 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.925781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.927441 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.927450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.930171 (+ 2721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.930189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.931791 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.931802 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.932026 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.932851 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.932878 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.934086 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.937754 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.937760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.937764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.938225 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.939450 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.939457 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.939464 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.939654 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.940445 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.940452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.940706 (+ 254us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.942329 (+ 1623us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.942730 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":973,"threads_started":1}
I20260621 06:58:29.954185 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.923012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.2.245.253:33397)
0621 06:58:29.923383 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.923388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.938853 (+ 15465us) server_negotiation.cc:407] Connection header received
0621 06:58:29.939115 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.939120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.939188 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.939250 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.940694 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.941823 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.943925 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.944603 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.944701 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.947669 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.947675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.947680 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.947713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.951209 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.952000 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.952005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.952009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.952164 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.952570 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.952574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.952578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.952917 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.953077 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.953565 (+ 488us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.953768 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260621 06:58:29.955644 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.922661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42124)
0621 06:58:29.938758 (+ 16097us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.938778 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.938976 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.939455 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.939459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.940151 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.940515 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.940534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.942046 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.942051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.943780 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.943793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.945006 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.945012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.945189 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.945960 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.945983 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.947478 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.951361 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.951366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.951370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.951887 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.952284 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.952288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.952292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.952467 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.953140 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.953144 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.953278 (+ 134us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.954822 (+ 1544us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.955121 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15941,"mutex_wait_us":33,"thread_start_us":15719,"threads_started":1}
WARNING: no policy specified for oryx/127.2.245.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.252@KRBTEST.COM" created.
Jun 21 06:58:29 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.254@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
I20260621 06:58:29.975626 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.951195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42128)
0621 06:58:29.956967 (+ 5772us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:29.957019 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:29.957537 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:29.958220 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:29.958228 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:29.959128 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:29.959431 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.959445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.960563 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.960570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.962295 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:29.962314 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.963435 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.963444 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:29.963697 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.964495 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:29.964527 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:29.967783 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:29.972224 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.972238 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.972246 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.972837 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.973329 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:29.973337 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:29.973343 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:29.973537 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:29.974313 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:29.974329 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:29.974555 (+ 226us) client_negotiation.cc:769] Sending connection context
0621 06:58:29.974796 (+ 241us) client_negotiation.cc:240] Negotiation successful
0621 06:58:29.975169 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5483,"thread_start_us":5324,"threads_started":1}
I20260621 06:58:29.975744 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:29.951681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.2.245.253:33397)
0621 06:58:29.954741 (+ 3060us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:29.954746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:29.957095 (+ 2349us) server_negotiation.cc:407] Connection header received
0621 06:58:29.957761 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:29.957767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:29.957847 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:29.957935 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:29.959614 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.960418 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.962456 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:29.963247 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:29.963411 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:29.967999 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:29.968007 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:29.968012 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:29.968056 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:29.972072 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.972986 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.972994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.973000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.973163 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:29.973670 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:29.973677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:29.973687 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:29.974140 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:29.974304 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:29.974967 (+ 663us) server_negotiation.cc:299] Negotiation successful
0621 06:58:29.975240 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2954,"thread_start_us":5554,"threads_started":1}
I20260621 06:58:29.978351 9164 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.245.253:33397: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8"
member_type: NON_VOTER
last_known_addr {
host: "127.2.245.253"
port: 33397
}
attrs {
promote: true
}
}
dest_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2"
cas_config_opid_index: -1
I20260621 06:58:29.979650 9164 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } }
I20260621 06:58:29.983585 9724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:29.984972 9667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:29.987560 9724 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.245.253:33397
I20260621 06:58:29.988729 9625 master.cc:537] Master@127.2.245.253:33397 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.252.keytab.
Entry for principal oryx/127.2.245.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.252.keytab.
I20260621 06:58:29.989351 9667 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260621 06:58:29.994737 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:39637
--webserver_interface=127.2.245.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.252.keytab
--principal=oryx/127.2.245.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 06:58:30.002620 9725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 06:58:30.006031 9151 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260621 06:58:30.007445 9625 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:30.007866 9625 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:30.007601 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.007987 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:30.008057 9625 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: stopping tablet replica
W20260621 06:58:30.008698 9151 consensus_peers.cc:487] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 -> Peer 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253:33397): Couldn't send request to peer 0dc56c3e4389491d9d909f0662822aa8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260621 06:58:30.015226 9625 master.cc:552] Master@127.2.245.253:33397 shutdown complete.
I20260621 06:58:30.015328 9625 master_runner.cc:305] Clearing existing system tablet
I20260621 06:58:30.015887 9625 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:30.016013 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:30.023618 9625 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.004s
I20260621 06:58:30.025063 9730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:30.025628 9625 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:30.025874 9625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 06:58:30.025955 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "0dc56c3e4389491d9d909f0662822aa8"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "6bff76dcb52b92831c8bb5442ce5466d"
server_key_iv: "e1bdc9d849d80c4c266308ad4a4bd024"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.029453 9625 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:30.033977 9625 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260621 06:58:30.034150 9625 log.cc:1196] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260621 06:58:30.034530 9625 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Deleting consensus metadata
I20260621 06:58:30.034786 9625 master_runner.cc:315] Copying system tablet from 127.2.245.254:38303
I20260621 06:58:30.037262 9625 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: tablet copy: Beginning tablet copy session from remote peer at address 127.2.245.254:38303
I20260621 06:58:30.049640 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.038053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.2.245.254:38303)
0621 06:58:30.038265 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.038274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.038732 (+ 458us) server_negotiation.cc:407] Connection header received
0621 06:58:30.038978 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.038984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.039073 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.039200 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.040002 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.040757 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.042065 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.042503 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.042645 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.044380 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.044388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.044397 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.044428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.047104 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.047700 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.047705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.047709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.047840 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.048190 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.048197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.048200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.048491 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.048646 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.049031 (+ 385us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.049247 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 06:58:30.049683 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.038003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36660)
0621 06:58:30.038614 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.038635 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.038795 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.039226 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.039231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.039643 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.039844 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.039853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.040914 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.040919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.041905 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.041916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.042651 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.042656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.042835 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.043526 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.043543 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.044172 (+ 629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.047228 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.047235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.047239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.047579 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.047945 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.047951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.047954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.048083 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.048612 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.048617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.048778 (+ 161us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.048946 (+ 168us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.049243 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":407,"threads_started":1}
I20260621 06:58:30.050793 9184 tablet_copy_service.cc:140] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0dc56c3e4389491d9d909f0662822aa8 ({username='oryx', principal='oryx/127.2.245.253@KRBTEST.COM'} at 127.0.0.1:36660)
I20260621 06:58:30.051091 9184 tablet_copy_service.cc:161] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0dc56c3e4389491d9d909f0662822aa8 at {username='oryx', principal='oryx/127.2.245.253@KRBTEST.COM'} at 127.0.0.1:36660: session id = 0dc56c3e4389491d9d909f0662822aa8-00000000000000000000000000000000
I20260621 06:58:30.054090 9184 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Tablet Copy: opened 0 blocks and 1 log segments
I20260621 06:58:30.055927 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260621 06:58:30.060730 9625 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: tablet copy: Starting download of 0 data blocks...
I20260621 06:58:30.061020 9625 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: tablet copy: Starting download of 1 WAL segments...
I20260621 06:58:30.063788 9625 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260621 06:58:30.067271 9184 tablet_copy_service.cc:342] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Request end of tablet copy session 0dc56c3e4389491d9d909f0662822aa8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.245.253@KRBTEST.COM'} at 127.0.0.1:36660
I20260621 06:58:30.067466 9184 tablet_copy_service.cc:433] P cb53aeb7e9f94cdaaff92e1a3acbfec2: ending tablet copy session 0dc56c3e4389491d9d909f0662822aa8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0dc56c3e4389491d9d909f0662822aa8
I20260621 06:58:30.077549 9625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:30.080083 9737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:30.080855 9738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:30.087904 9625 server_base.cc:1034] running on GCE node
W20260621 06:58:30.091467 9740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:30.091939 9625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:30.092661 9625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:30.093876 9625 hybrid_clock.cc:648] HybridClock initialized: now 1782025110093794 us; error 51 us; skew 500 ppm
I20260621 06:58:30.094069 9625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:30.095944 9625 init.cc:377] Logged in from keytab as oryx/127.2.245.253@KRBTEST.COM (short username oryx)
I20260621 06:58:30.097160 9625 webserver.cc:460] Webserver started at http://127.2.245.253:35461/ using document root <none> and password file <none>
I20260621 06:58:30.097540 9625 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:30.097631 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:30.100100 9625 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260621 06:58:30.101668 9747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:30.102110 9625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260621 06:58:30.102248 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "0dc56c3e4389491d9d909f0662822aa8"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "6bff76dcb52b92831c8bb5442ce5466d"
server_key_iv: "e1bdc9d849d80c4c266308ad4a4bd024"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.102404 9625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:30.117063 9625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:30.120579 9625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:30.139966 9625 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.253:33397
I20260621 06:58:30.139988 9798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.253:33397 every 8 connection(s)
I20260621 06:58:30.140848 9625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
I20260621 06:58:30.147789 9799 sys_catalog.cc:263] Verifying existing consensus state
I20260621 06:58:30.150846 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Bootstrap starting.
I20260621 06:58:30.163967 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 06:58:30.164403 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: Bootstrap complete.
I20260621 06:58:30.165354 9799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } }
I20260621 06:58:30.165592 9799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0dc56c3e4389491d9d909f0662822aa8, State: Initialized, Role: LEARNER
I20260621 06:58:30.165797 9799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } }
I20260621 06:58:30.166791 9802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.167165 9799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:30.167200 9802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: LEARNER
I20260621 06:58:30.177872 9813 catalog_manager.cc:1259] Loaded cluster ID: e862c6e13ebc41fdb7b68be0dee74ebf
I20260621 06:58:30.177990 9813 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: loading cluster ID for follower catalog manager: success
I20260621 06:58:30.180609 9813 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: acquiring CA information for follower catalog manager: success
I20260621 06:58:30.182420 9813 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 06:58:30.340883 9620 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
I20260621 06:58:30.416435 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.409121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42130)
0621 06:58:30.409394 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.409417 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.409577 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.410887 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.410893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.410923 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:30.411148 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.411158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.412562 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.412568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.413780 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.413793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.415548 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.415554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.415795 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.415869 (+ 74us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.415982 (+ 113us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.416139 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260621 06:58:30.416507 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.409262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.2.245.253:33397)
0621 06:58:30.410539 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.410544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.410560 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:30.410619 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.410622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.410700 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.410856 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:30.411320 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.412405 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.413931 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.415369 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.415511 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.415793 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.416012 (+ 219us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.416155 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":1012,"threads_started":1}
I20260621 06:58:30.418277 9774 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260621 06:58:30.419096 9802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb53aeb7e9f94cdaaff92e1a3acbfec2. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.419384 9802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: LEARNER
W20260621 06:58:30.463858 9726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:30.464233 9726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:30.464351 9726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:30.476176 9726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:30.476307 9726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:30.476392 9726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:30.476469 9726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:30.476523 9726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:30.493094 9726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.252.keytab
--principal=oryx/127.2.245.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:39637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.245.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:30.494716 9726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:30.496357 9726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:30.510406 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:30.510425 9820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:30.511123 9822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:30.511359 9726 server_base.cc:1034] running on GCE node
I20260621 06:58:30.512117 9726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:30.513535 9726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:30.514760 9726 hybrid_clock.cc:648] HybridClock initialized: now 1782025110514721 us; error 61 us; skew 500 ppm
I20260621 06:58:30.515179 9726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:30.518261 9726 init.cc:377] Logged in from keytab as oryx/127.2.245.252@KRBTEST.COM (short username oryx)
I20260621 06:58:30.519716 9726 webserver.cc:460] Webserver started at http://127.2.245.252:37437/ using document root <none> and password file <none>
I20260621 06:58:30.520386 9726 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:30.520540 9726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:30.520828 9726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:30.523802 9726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/instance:
uuid: "8add484e2fc04e7e8f57750de2239623"
format_stamp: "Formatted at 2026-06-21 06:58:30 on dist-test-slave-9zdv"
server_key: "953bc333e302a9f45b2b7ea5ae6d1937"
server_key_iv: "4eaf4cd26523c91666769b3de16583e2"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.524565 9726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal/instance:
uuid: "8add484e2fc04e7e8f57750de2239623"
format_stamp: "Formatted at 2026-06-21 06:58:30 on dist-test-slave-9zdv"
server_key: "953bc333e302a9f45b2b7ea5ae6d1937"
server_key_iv: "4eaf4cd26523c91666769b3de16583e2"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.529876 9726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:58:30.533690 9829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:30.535027 9726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:30.535202 9726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "8add484e2fc04e7e8f57750de2239623"
format_stamp: "Formatted at 2026-06-21 06:58:30 on dist-test-slave-9zdv"
server_key: "953bc333e302a9f45b2b7ea5ae6d1937"
server_key_iv: "4eaf4cd26523c91666769b3de16583e2"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.535401 9726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:30.559075 9726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:30.564069 9726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:30.564352 9726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:30.585054 9726 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.252:39637
I20260621 06:58:30.585137 9880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.252:39637 every 8 connection(s)
I20260621 06:58:30.586367 9726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
I20260621 06:58:30.589938 9881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:30.592605 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9726
I20260621 06:58:30.599165 9881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.252@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:30.616137 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.594905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.2.245.252:39637)
0621 06:58:30.595693 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.595708 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.595751 (+ 43us) server_negotiation.cc:407] Connection header received
0621 06:58:30.597166 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.597188 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.597591 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.597935 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.599393 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.600724 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.602378 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.603090 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.603261 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.608568 (+ 5307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.608576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.608586 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.608616 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.611807 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.612435 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.612463 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.612473 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.612670 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.613099 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.613104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.613106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.613791 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.614100 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.614471 (+ 371us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.614902 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":329,"threads_started":1}
I20260621 06:58:30.617306 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.601456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36674)
0621 06:58:30.602142 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.602194 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.602329 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.603323 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.603329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.604149 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.604445 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.604459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.605551 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.605557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.607174 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.607185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.608121 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.608129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.608320 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.608983 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.608999 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.611557 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.615060 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.615069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.615074 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.615463 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.615831 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.615834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.615837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.615958 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.616548 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.616553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.616687 (+ 134us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.616838 (+ 151us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.617048 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":265,"threads_started":1}
I20260621 06:58:30.617511 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.601656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.2.245.254:38303)
0621 06:58:30.602821 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.602825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.602847 (+ 22us) server_negotiation.cc:407] Connection header received
0621 06:58:30.602892 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.602896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.602998 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.603111 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.604608 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.605244 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.607311 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.607847 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.608006 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.611751 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.611762 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.611767 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.611805 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.614902 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.615580 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.615586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.615589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.615718 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.616083 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.616090 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.616095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.616435 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.616574 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.616953 (+ 379us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.617181 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":961,"threads_started":1}
I20260621 06:58:30.618736 9881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.252@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
I20260621 06:58:30.630932 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.619584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42132)
0621 06:58:30.619763 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.619782 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.619921 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.620233 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.620237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.620579 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.621102 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.621110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.622052 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.622060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.622993 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.623010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.623663 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.623667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.623826 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.624358 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.624372 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.626093 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.628694 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.628700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.628703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.629059 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.629429 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.629436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.629439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.629569 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.630043 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.630048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.630187 (+ 139us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.630322 (+ 135us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.630612 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 06:58:30.631019 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.619568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.2.245.253:33397)
0621 06:58:30.619765 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.619772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.619864 (+ 92us) server_negotiation.cc:407] Connection header received
0621 06:58:30.620046 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.620051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.620121 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.620220 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.621257 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.621931 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.623147 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.623537 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.623658 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.626260 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.626270 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.626273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.626302 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.628578 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.629192 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.629198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.629202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.629323 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.629671 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.629678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.629681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.629934 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.630081 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.630426 (+ 345us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.630656 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 06:58:30.631994 9881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.252@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
I20260621 06:58:30.642799 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.632567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.0.0.1:42330)
0621 06:58:30.632751 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.632765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.632837 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.633103 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.633105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.633372 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.633495 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.633501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.634266 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.634268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.635148 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.635159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.635745 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.635749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.635891 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.636444 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.636459 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.638262 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.640999 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.641002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.641005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.641332 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.641618 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.641621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.641623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.641750 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.642166 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.642169 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.642278 (+ 109us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.642379 (+ 101us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.642541 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 06:58:30.642915 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.632630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.2.245.252:39637)
0621 06:58:30.632774 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.632778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.632795 (+ 17us) server_negotiation.cc:407] Connection header received
0621 06:58:30.632941 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.632944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.632991 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.633059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.633592 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.634173 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.635256 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.635632 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.635725 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.638408 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.638412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.638414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.638437 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.640884 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.641437 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.641441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.641443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.641533 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.641831 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.641834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.641836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.642076 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.642172 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.642449 (+ 277us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.642634 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":52}
I20260621 06:58:30.647543 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Bootstrap starting.
I20260621 06:58:30.650589 9881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:30.651605 9881 log.cc:824] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:30.653927 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: No bootstrap required, opened a new log
I20260621 06:58:30.661988 9881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } }
I20260621 06:58:30.662262 9881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:30.662388 9881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8add484e2fc04e7e8f57750de2239623, State: Initialized, Role: FOLLOWER
I20260621 06:58:30.663040 9881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } }
I20260621 06:58:30.663755 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } } }
I20260621 06:58:30.664080 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:30.664814 9881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:30.671350 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9225
W20260621 06:58:30.675194 9899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 06:58:30.675577 9899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 06:58:30.678625 9726 master_runner.cc:428] Detected that this master 8add484e2fc04e7e8f57750de2239623 is joining an existing cluster
I20260621 06:58:30.678747 9726 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.245.252:39637
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.193@KRBTEST.COM
I20260621 06:58:30.696915 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.672707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.2.245.193:43637)
0621 06:58:30.677173 (+ 4466us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.677179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.677204 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:58:30.677269 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.677275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.677450 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.678135 (+ 685us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.679748 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.681207 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.682409 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.683021 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.683192 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.688604 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.688618 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.688636 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.688692 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.692748 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.693581 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.693590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.693594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.693759 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.694214 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.694219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.694224 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.694665 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.695015 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.696214 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.696475 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4307,"thread_start_us":4173,"threads_started":1}
I20260621 06:58:30.700050 9348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:30.711685 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.683828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.2.245.254:38303)
0621 06:58:30.683988 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.683994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.684053 (+ 59us) server_negotiation.cc:407] Connection header received
0621 06:58:30.684299 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.684307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.684387 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.684458 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.686162 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.687617 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.690429 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.691640 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.691778 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.694913 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.694923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.694930 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.695010 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.699417 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.707456 (+ 8039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.707466 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.707472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.707676 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.708756 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.708765 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.708771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.709243 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.709397 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.710807 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.711187 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 06:58:30.715161 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.683411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36682)
0621 06:58:30.683967 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.683985 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.684152 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.684666 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.684671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.685463 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.685950 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.685967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.687969 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.687977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.690058 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.690075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.692192 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.692202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.692421 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.693178 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.693209 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.694626 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.705744 (+ 11118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.705752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.705758 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.706407 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.707910 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.707917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.707923 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.708162 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.709551 (+ 1389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.709558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.709869 (+ 311us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.712319 (+ 2450us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.714183 (+ 1864us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":586,"threads_started":1}
I20260621 06:58:30.717167 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.684678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.2.245.252:39637)
0621 06:58:30.684855 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.684860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.699596 (+ 14736us) server_negotiation.cc:407] Connection header received
0621 06:58:30.699812 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.699817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.699888 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.699952 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.701680 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.702749 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.704943 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.705643 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.705740 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.708293 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.708299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.708304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.708335 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.712311 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.713219 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.713223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.713227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.713397 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.713890 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.713897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.713902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.714753 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.715005 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.716586 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.716810 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":22272}
I20260621 06:58:30.718088 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.684410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.0.0.1:42340)
0621 06:58:30.699506 (+ 15096us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.699525 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.699693 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.700128 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.700133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.701238 (+ 1105us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.701543 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.701554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.703135 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.703145 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.704792 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.704807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.706081 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.706086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.706248 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.706934 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.706973 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.708122 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.712488 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.712492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.712496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.713091 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.713536 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.713540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.713544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.713732 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.715060 (+ 1328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.715065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.715227 (+ 162us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.717347 (+ 2120us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.717676 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14939,"thread_start_us":14746,"threads_started":1}
I20260621 06:58:30.731455 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.690084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.2.245.253:33397)
0621 06:58:30.690292 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.690299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.713556 (+ 23257us) server_negotiation.cc:407] Connection header received
0621 06:58:30.713758 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.713768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.713923 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.714043 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.718307 (+ 4264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.719441 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.721539 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.722190 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.722292 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.724629 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.724636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.724641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.724679 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.728018 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.728800 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.728805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.728809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.728985 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.729501 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.729507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.729510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.729833 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.729974 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.730580 (+ 606us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.730806 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 06:58:30.732402 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.691008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:42134)
0621 06:58:30.713417 (+ 22409us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.713444 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.713599 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.714240 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.714247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.715979 (+ 1732us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.716282 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.716293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.719625 (+ 3332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.719630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.721374 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.721393 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.722584 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.722591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.722774 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.723362 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.723382 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.724415 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.728191 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.728196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.728200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.728657 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.729140 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.729145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.729149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.729366 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.730075 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.730080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.730280 (+ 200us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.731686 (+ 1406us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.731985 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22277,"thread_start_us":22141,"threads_started":1}
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.254@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
I20260621 06:58:30.746783 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.730021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.0.0.1:42352)
0621 06:58:30.730362 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.730391 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.730484 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.731378 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.731383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.731825 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.732017 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.732027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.733816 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.733821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.735311 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.735328 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.737061 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.737067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.737222 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.737823 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.737853 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.740632 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.744267 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.744272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.744275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.744703 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.745087 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.745091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.745095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.745231 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.745820 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.745825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.745993 (+ 168us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.746118 (+ 125us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.746405 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260621 06:58:30.747714 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.730748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.2.245.252:39637)
0621 06:58:30.730900 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.730905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.730923 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:58:30.731034 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.731039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.731129 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.731208 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.732438 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.733461 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.735481 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.736731 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.736837 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.740818 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.740824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.740829 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.740873 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.744131 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.744821 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.744826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.744829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.744958 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.745342 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.745346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.745349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.745650 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.745768 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.747212 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.747406 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 06:58:30.749223 9164 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.245.252:39637: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8add484e2fc04e7e8f57750de2239623"
member_type: NON_VOTER
last_known_addr {
host: "127.2.245.252"
port: 39637
}
attrs {
promote: true
}
}
dest_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2"
cas_config_opid_index: 5
I20260621 06:58:30.750080 9164 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
I20260621 06:58:30.752466 9723 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8add484e2fc04e7e8f57750de2239623 (127.2.245.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.755844 9774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260621 06:58:30.756287 9856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:30.759757 9856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260621 06:58:30.762028 9910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260621 06:58:30.768769 9911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:30.774067 9774 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8add484e2fc04e7e8f57750de2239623 (127.2.245.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.780573 9802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.780908 9802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: LEARNER
I20260621 06:58:30.783195 9723 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.245.252:39637
I20260621 06:58:30.783764 9723 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: attempting to promote NON_VOTER 0dc56c3e4389491d9d909f0662822aa8 to VOTER
I20260621 06:58:30.783887 9726 master.cc:537] Master@127.2.245.252:39637 shutting down...
I20260621 06:58:30.786494 9856 raft_consensus.cc:858] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
W20260621 06:58:30.786932 9856 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
I20260621 06:58:30.785380 9723 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
I20260621 06:58:30.794682 9910 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.795183 9910 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:30.797276 9911 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.797619 9911 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:30.803828 9774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 LEARNER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260621 06:58:30.804186 9149 consensus_peers.cc:487] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 -> Peer 8add484e2fc04e7e8f57750de2239623 (127.2.245.252:39637): Couldn't send request to peer 8add484e2fc04e7e8f57750de2239623. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260621 06:58:30.804497 9912 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260621 06:58:30.807124 9912 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.811214 9910 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.811470 9910 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:30.811872 9912 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.812278 9912 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:30.813174 9774 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.817540 9802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:30.817835 9802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:30.826787 9856 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.831279 9921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.831622 9921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260621 06:58:30.832060 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb53aeb7e9f94cdaaff92e1a3acbfec2. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.832283 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260621 06:58:30.834328 9917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: NON_VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: true } } }
I20260621 06:58:30.834569 9917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260621 06:58:30.834798 9726 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260621 06:58:30.835326 9726 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260621 06:58:30.835512 9726 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: stopping tablet replica
I20260621 06:58:30.843787 9726 master.cc:552] Master@127.2.245.252:39637 shutdown complete.
I20260621 06:58:30.843875 9726 master_runner.cc:305] Clearing existing system tablet
I20260621 06:58:30.844257 9726 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:30.844408 9726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:30.847626 9726 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260621 06:58:30.849195 9923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:30.849932 9726 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:30.850327 9726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 06:58:30.850471 9726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "8add484e2fc04e7e8f57750de2239623"
format_stamp: "Formatted at 2026-06-21 06:58:30 on dist-test-slave-9zdv"
server_key: "953bc333e302a9f45b2b7ea5ae6d1937"
server_key_iv: "4eaf4cd26523c91666769b3de16583e2"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.855238 9726 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:30.862263 9726 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260621 06:58:30.862519 9726 log.cc:1196] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260621 06:58:30.863293 9726 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Deleting consensus metadata
I20260621 06:58:30.863590 9726 master_runner.cc:315] Copying system tablet from 127.2.245.254:38303
I20260621 06:58:30.866235 9726 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: tablet copy: Beginning tablet copy session from remote peer at address 127.2.245.254:38303
I20260621 06:58:30.876773 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9225
I20260621 06:58:30.888775 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.867233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.2.245.254:38303)
0621 06:58:30.867472 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:30.867478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:30.871844 (+ 4366us) server_negotiation.cc:407] Connection header received
0621 06:58:30.872176 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:30.872185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:30.872297 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:30.872400 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:30.874400 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.875738 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.877760 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.878557 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.878672 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.881625 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:30.881634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:30.881642 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:30.881687 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:30.885334 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.886146 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.886154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.886160 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.886305 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:30.886800 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:30.886809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:30.886815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:30.887232 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:30.887364 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:30.888043 (+ 679us) server_negotiation.cc:299] Negotiation successful
0621 06:58:30.888299 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260621 06:58:30.890000 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:30.867106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:36692)
0621 06:58:30.871699 (+ 4593us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:30.871720 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:30.871984 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:30.872669 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:30.872677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:30.873808 (+ 1131us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:30.874153 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.874168 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.875937 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.875942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.877581 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:30.877599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:30.879082 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:30.879090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:30.879344 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:30.880101 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:30.880129 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:30.881409 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:30.885502 (+ 4093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.885509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.885516 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.886001 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.886457 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:30.886467 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:30.886473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:30.886675 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:30.887485 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:30.887494 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:30.887740 (+ 246us) client_negotiation.cc:769] Sending connection context
0621 06:58:30.889033 (+ 1293us) client_negotiation.cc:240] Negotiation successful
0621 06:58:30.889426 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4436,"thread_start_us":4178,"threads_started":1}
I20260621 06:58:30.890911 9184 tablet_copy_service.cc:140] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8add484e2fc04e7e8f57750de2239623 ({username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:36692)
I20260621 06:58:30.891098 9184 tablet_copy_service.cc:161] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8add484e2fc04e7e8f57750de2239623 at {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:36692: session id = 8add484e2fc04e7e8f57750de2239623-00000000000000000000000000000000
I20260621 06:58:30.894659 9184 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Tablet Copy: opened 0 blocks and 1 log segments
I20260621 06:58:30.896674 9726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260621 06:58:30.899461 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:43637
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:38303
--webserver_port=35795
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 06:58:30.902199 9726 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: tablet copy: Starting download of 0 data blocks...
I20260621 06:58:30.902518 9726 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: tablet copy: Starting download of 1 WAL segments...
I20260621 06:58:30.904978 9726 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260621 06:58:30.908864 9184 tablet_copy_service.cc:342] P cb53aeb7e9f94cdaaff92e1a3acbfec2: Request end of tablet copy session 8add484e2fc04e7e8f57750de2239623-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:36692
I20260621 06:58:30.909013 9184 tablet_copy_service.cc:433] P cb53aeb7e9f94cdaaff92e1a3acbfec2: ending tablet copy session 8add484e2fc04e7e8f57750de2239623-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8add484e2fc04e7e8f57750de2239623
I20260621 06:58:30.921800 9726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:30.924649 9931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:30.925860 9932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:30.932629 9934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:30.936237 9726 server_base.cc:1034] running on GCE node
I20260621 06:58:30.936573 9726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:30.939181 9726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:30.940397 9726 hybrid_clock.cc:648] HybridClock initialized: now 1782025110940350 us; error 34 us; skew 500 ppm
I20260621 06:58:30.940639 9726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:30 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025110, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:30.942862 9726 init.cc:377] Logged in from keytab as oryx/127.2.245.252@KRBTEST.COM (short username oryx)
I20260621 06:58:30.947233 9726 webserver.cc:460] Webserver started at http://127.2.245.252:37335/ using document root <none> and password file <none>
I20260621 06:58:30.947561 9726 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:30.947659 9726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:30.950615 9726 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260621 06:58:30.955363 9941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:30.955804 9726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260621 06:58:30.955973 9726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "8add484e2fc04e7e8f57750de2239623"
format_stamp: "Formatted at 2026-06-21 06:58:30 on dist-test-slave-9zdv"
server_key: "953bc333e302a9f45b2b7ea5ae6d1937"
server_key_iv: "4eaf4cd26523c91666769b3de16583e2"
server_key_version: "encryptionkey@0"
I20260621 06:58:30.956234 9726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:30.992890 9726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:30.997156 9726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:31.033643 9726 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.252:39637
I20260621 06:58:31.033701 9992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.252:39637 every 8 connection(s)
I20260621 06:58:31.034632 9726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
I20260621 06:58:31.040076 9993 sys_catalog.cc:263] Verifying existing consensus state
I20260621 06:58:31.044402 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Bootstrap starting.
I20260621 06:58:31.060357 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 06:58:31.060914 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Bootstrap complete.
I20260621 06:58:31.062027 9993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
I20260621 06:58:31.062397 9993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8add484e2fc04e7e8f57750de2239623, State: Initialized, Role: LEARNER
I20260621 06:58:31.062762 9993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } }
I20260621 06:58:31.069833 9996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:31.070276 9996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: LEARNER
I20260621 06:58:31.070963 9993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:31.091941 10007 catalog_manager.cc:1259] Loaded cluster ID: e862c6e13ebc41fdb7b68be0dee74ebf
I20260621 06:58:31.092079 10007 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: loading cluster ID for follower catalog manager: success
I20260621 06:58:31.095504 10007 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: acquiring CA information for follower catalog manager: success
I20260621 06:58:31.097991 10007 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260621 06:58:31.283972 9930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:31.284245 9930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:31.284428 9930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:31.291723 9930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:31.291826 9930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:31.291949 9930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:31.301640 9930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:43637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=35795
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:31.302995 9930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:31.304347 9930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:31.318775 10013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:31.319038 10012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:31.319182 10015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:31.320748 9930 server_base.cc:1034] running on GCE node
I20260621 06:58:31.321408 9930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:31.322538 9930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:31.324023 9930 hybrid_clock.cc:648] HybridClock initialized: now 1782025111323954 us; error 58 us; skew 500 ppm
I20260621 06:58:31.324100 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.315208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.0.0.1:42358)
0621 06:58:31.316418 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:31.316435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:31.316554 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:31.317070 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:31.317076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:31.317123 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:31.317355 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.317367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.319457 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.319462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.321246 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.321260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.322813 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.322819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.323039 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.323077 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 06:58:31.323200 (+ 123us) client_negotiation.cc:240] Negotiation successful
0621 06:58:31.323347 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"thread_start_us":996,"threads_started":1}
I20260621 06:58:31.324112 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.315377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.2.245.252:39637)
0621 06:58:31.316386 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:31.316393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:31.316523 (+ 130us) server_negotiation.cc:407] Connection header received
0621 06:58:31.316730 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:31.316735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:31.316831 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:31.316971 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:31.317514 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.319220 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.321403 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.322634 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.322775 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.323049 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:31.323223 (+ 174us) server_negotiation.cc:299] Negotiation successful
0621 06:58:31.323349 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":569,"threads_started":1}
I20260621 06:58:31.324445 9930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:58:31.325088 9968 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260621 06:58:31.325537 9996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb53aeb7e9f94cdaaff92e1a3acbfec2. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:31.325825 9996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: LEARNER
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:31.327795 9930 init.cc:377] Logged in from keytab as oryx/127.2.245.193@KRBTEST.COM (short username oryx)
I20260621 06:58:31.329200 9930 webserver.cc:460] Webserver started at http://127.2.245.193:35795/ using document root <none> and password file <none>
I20260621 06:58:31.329875 9930 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:31.329985 9930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:31.335358 9930 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 06:58:31.337919 10024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.339211 9930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:31.339417 9930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "c782c8644c7440eb9a3befd68a4fc500"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "63428ed120e457fe8e3a8b794b07fa37"
server_key_iv: "20aa97fde14168b5c3f6316e2d708e7c"
server_key_version: "encryptionkey@0"
I20260621 06:58:31.340361 9930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:31.353205 9930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:31.357143 9930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:31.357478 9930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:31.358531 9930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:31.359939 9930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:31.360023 9930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.360176 9930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:31.360268 9930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.395396 9930 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:43637
I20260621 06:58:31.395437 10136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:43637 every 8 connection(s)
I20260621 06:58:31.396844 9930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:31.405951 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 9930
I20260621 06:58:31.406379 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9359
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.194@KRBTEST.COM
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
I20260621 06:58:31.436386 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.401316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:54861 (local address 127.2.245.254:38303)
0621 06:58:31.415365 (+ 14049us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:31.415375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:31.415401 (+ 26us) server_negotiation.cc:407] Connection header received
0621 06:58:31.415476 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:31.415485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:31.415578 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:31.415714 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:31.418142 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.419258 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.421930 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.422560 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.422680 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.427793 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:31.427801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:31.427806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:31.427851 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:31.431580 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.432584 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.432591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.432595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.432738 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.433595 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.433600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.433603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.433978 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:31.434113 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:31.435718 (+ 1605us) server_negotiation.cc:299] Negotiation successful
0621 06:58:31.435968 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13885,"thread_start_us":14730,"threads_started":1}
I20260621 06:58:31.437193 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.403609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:55277 (local address 127.2.245.252:39637)
0621 06:58:31.403808 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:31.403814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:31.418630 (+ 14816us) server_negotiation.cc:407] Connection header received
0621 06:58:31.418776 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:31.418783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:31.418864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:31.419003 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:31.420323 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.421370 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.423040 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.425105 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.425240 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.430270 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:31.430277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:31.430281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:31.430322 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:31.434123 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435034 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.435039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.435042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.435215 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435678 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.435685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.435688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.436017 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:31.436149 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:31.436623 (+ 474us) server_negotiation.cc:299] Negotiation successful
0621 06:58:31.436869 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 06:58:31.437780 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.408068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.2.245.194:34131)
0621 06:58:31.413931 (+ 5863us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:31.413945 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:31.414679 (+ 734us) server_negotiation.cc:407] Connection header received
0621 06:58:31.414903 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:31.414912 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:31.415236 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:31.417455 (+ 2219us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:31.417642 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.419047 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.420985 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.421824 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.421947 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.429542 (+ 7595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:31.429562 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:31.429585 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:31.429641 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:31.434325 (+ 4684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435267 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.435275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.435279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.435436 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435881 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.435888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.435892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.436288 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:31.436592 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:31.437106 (+ 514us) server_negotiation.cc:299] Negotiation successful
0621 06:58:31.437377 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5495,"thread_start_us":5274,"threads_started":1}
I20260621 06:58:31.438367 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.401616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:47879 (local address 127.2.245.253:33397)
0621 06:58:31.412749 (+ 11133us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:31.412758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:31.414053 (+ 1295us) server_negotiation.cc:407] Connection header received
0621 06:58:31.414145 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:31.414150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:31.414244 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:31.414356 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:31.419556 (+ 5200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.420713 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.424206 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.424809 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.425095 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.431121 (+ 6026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:31.431125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:31.431129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:31.431188 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:31.435096 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435968 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.435973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.435978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.436128 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:31.436663 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:31.436668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:31.436671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:31.437001 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:31.437129 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:31.437893 (+ 764us) server_negotiation.cc:299] Negotiation successful
0621 06:58:31.438110 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10967,"thread_start_us":10837,"threads_started":1}
I20260621 06:58:31.439379 9482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:31.439455 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.403920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.2.245.193:55277)
0621 06:58:31.418501 (+ 14581us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:31.418518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:31.418666 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:31.419181 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:31.419186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:31.419831 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:31.420151 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.420161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.421468 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.421475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.422845 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.422868 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.425522 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.425528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.426514 (+ 986us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.427180 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:31.427205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:31.430077 (+ 2872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:31.434278 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.434284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.434287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.434821 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.435370 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.435377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.435545 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.436227 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:31.436231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:31.436368 (+ 137us) client_negotiation.cc:769] Sending connection context
0621 06:58:31.437685 (+ 1317us) client_negotiation.cc:240] Negotiation successful
0621 06:58:31.438116 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14463,"thread_start_us":9021,"threads_started":1}
I20260621 06:58:31.439522 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.400699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.2.245.193:47879)
0621 06:58:31.413289 (+ 12590us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:31.413315 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:31.413478 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:31.415222 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:31.415235 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:31.418999 (+ 3764us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:31.419339 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.419350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.420914 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.420919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.424022 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.424035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.425367 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.425375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.425599 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.426646 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:31.426675 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:31.430822 (+ 4147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:31.435250 (+ 4428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.435257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.435263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.435825 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.436237 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.436241 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.436244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.436374 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.437188 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:31.437194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:31.437350 (+ 156us) client_negotiation.cc:769] Sending connection context
0621 06:58:31.437463 (+ 113us) client_negotiation.cc:240] Negotiation successful
0621 06:58:31.438760 (+ 1297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12403,"mutex_wait_us":2704,"spinlock_wait_cycles":54016,"thread_start_us":17386,"threads_started":1}
I20260621 06:58:31.439550 10141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.400699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:54861)
0621 06:58:31.401697 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:31.401759 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:31.405236 (+ 3477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:31.415947 (+ 10711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:31.415955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:31.416783 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:31.417895 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.417931 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.419499 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.419504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.421767 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:31.421785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:31.422985 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:31.422993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:31.423280 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:31.424022 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:31.424051 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:31.427484 (+ 3433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:31.431746 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.431758 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.431781 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.432440 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.432867 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:31.432871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:31.432875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:31.433454 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:31.434202 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:31.434210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:31.434702 (+ 492us) client_negotiation.cc:769] Sending connection context
0621 06:58:31.435114 (+ 412us) client_negotiation.cc:240] Negotiation successful
0621 06:58:31.437739 (+ 2625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":2,"thread_start_us":1029,"threads_started":1}
I20260621 06:58:31.441169 10137 heartbeater.cc:347] Connected to a master server at 127.2.245.252:39637
I20260621 06:58:31.441486 10137 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:31.441648 10138 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:31.441821 10138 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:31.442297 10137 heartbeater.cc:511] Master 127.2.245.252:39637 requested a full tablet report, sending...
I20260621 06:58:31.442447 10138 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
I20260621 06:58:31.442857 10139 heartbeater.cc:347] Connected to a master server at 127.2.245.253:33397
I20260621 06:58:31.443002 10139 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:31.443403 10139 heartbeater.cc:511] Master 127.2.245.253:33397 requested a full tablet report, sending...
I20260621 06:58:31.445513 9764 ts_manager.cc:194] Registered new tserver with Master: c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637)
I20260621 06:58:31.445513 9958 ts_manager.cc:194] Registered new tserver with Master: c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637)
I20260621 06:58:31.447714 9164 ts_manager.cc:194] Re-registered known tserver with Master: c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637)
I20260621 06:58:31.448534 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:54861
I20260621 06:58:31.525782 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9359
I20260621 06:58:31.541733 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:34131
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:38303
--webserver_port=43451
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 06:58:31.784250 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: NON_VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: true } } }
I20260621 06:58:31.784631 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:31.786077 10151 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: attempting to promote NON_VOTER 8add484e2fc04e7e8f57750de2239623 to VOTER
I20260621 06:58:31.786643 10151 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } }
I20260621 06:58:31.788645 9774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260621 06:58:31.788684 9968 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 LEARNER]: Refusing update from remote peer cb53aeb7e9f94cdaaff92e1a3acbfec2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260621 06:58:31.789239 10150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260621 06:58:31.789685 9723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260621 06:58:31.792107 9723 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8add484e2fc04e7e8f57750de2239623 (127.2.245.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.792865 9774 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8add484e2fc04e7e8f57750de2239623 (127.2.245.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.796191 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.796440 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:31.796677 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.796952 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:31.797196 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.797418 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:31.797101 9968 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8add484e2fc04e7e8f57750de2239623 (127.2.245.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.801241 9996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.801180 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:31.801496 9996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:31.801522 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: FOLLOWER
W20260621 06:58:31.878340 10149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:31.878749 10149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:31.878923 10149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:31.886085 10149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:31.886231 10149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:31.886415 10149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:31.896160 10149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:34131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=43451
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:31.897554 10149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:31.898869 10149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:31.912456 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:31.912621 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:31.912770 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:31.913468 10149 server_base.cc:1034] running on GCE node
I20260621 06:58:31.914100 10149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:31.915102 10149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:31.916292 10149 hybrid_clock.cc:648] HybridClock initialized: now 1782025111916236 us; error 58 us; skew 500 ppm
I20260621 06:58:31.916579 10149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:31 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:31.919667 10149 init.cc:377] Logged in from keytab as oryx/127.2.245.194@KRBTEST.COM (short username oryx)
I20260621 06:58:31.921072 10149 webserver.cc:460] Webserver started at http://127.2.245.194:43451/ using document root <none> and password file <none>
I20260621 06:58:31.921768 10149 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:31.921911 10149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:31.927131 10149 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:31.929710 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.931218 10149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:31.931347 10149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "10ea24b03a784601b4d2b9c9dca562db"
format_stamp: "Formatted at 2026-06-21 06:58:28 on dist-test-slave-9zdv"
server_key: "338071caf3368d1d122a74bc83041948"
server_key_iv: "4fdf150ed1ed5f87ae6b457b08b39e74"
server_key_version: "encryptionkey@0"
I20260621 06:58:31.932354 10149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:31.945312 10149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:31.949298 10149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:31.949533 10149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:31.950403 10149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:31.951817 10149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:31.951906 10149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.951984 10149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:31.952036 10149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:31.985688 10149 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:34131
I20260621 06:58:31.985772 10281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:34131 every 8 connection(s)
I20260621 06:58:31.987223 10149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:31.992489 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10149
I20260621 06:58:31.992820 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9492
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.195@KRBTEST.COM
I20260621 06:58:32.045233 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.993991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52862 (local address 127.2.245.195:33905)
0621 06:58:32.003369 (+ 9378us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.003383 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.019058 (+ 15675us) server_negotiation.cc:407] Connection header received
0621 06:58:32.019186 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.019199 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.019549 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.019751 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.021944 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.024023 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.026525 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.028149 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.028277 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.036589 (+ 8312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.036605 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.036621 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.036662 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.040962 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.041870 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.041881 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.041885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.042044 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.042498 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.042505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.042509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.043182 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.043487 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.044156 (+ 669us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.044740 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9129,"thread_start_us":8583,"threads_started":1}
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025111, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:32.047947 9615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:32.048676 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.990867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.2.245.194:59165)
0621 06:58:32.017176 (+ 26309us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.017247 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.020102 (+ 2855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.023271 (+ 3169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.023288 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.024351 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.025382 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.025424 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.027843 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.027851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.030449 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.030466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.031697 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.031703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.031977 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.032767 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.032806 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.035953 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.041647 (+ 5694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.041655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.041673 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.042296 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.043662 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.043666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.043669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.043915 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.044538 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.044545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.045106 (+ 561us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.046126 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.047315 (+ 1189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25764,"thread_start_us":30554,"threads_started":1}
I20260621 06:58:32.050117 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.989993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59165 (local address 127.2.245.252:39637)
0621 06:58:32.022396 (+ 32403us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.022419 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.022444 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:58:32.022516 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.022524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.022615 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.025625 (+ 3010us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.025839 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.027278 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.030653 (+ 3375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.031296 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.031423 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.036220 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.036227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.036232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.036268 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.041163 (+ 4895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.042707 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.042712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.042716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.042829 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.044058 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.044062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.044066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.044391 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.049389 (+ 4998us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.049521 (+ 132us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.049736 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32255,"thread_start_us":32748,"threads_started":1}
I20260621 06:58:32.051285 10282 heartbeater.cc:347] Connected to a master server at 127.2.245.252:39637
I20260621 06:58:32.051625 10282 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.052865 10282 heartbeater.cc:511] Master 127.2.245.252:39637 requested a full tablet report, sending...
I20260621 06:58:32.054699 9958 ts_manager.cc:194] Registered new tserver with Master: 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131)
I20260621 06:58:32.057664 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.022213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:47585 (local address 127.2.245.254:38303)
0621 06:58:32.038220 (+ 16007us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.038228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.038248 (+ 20us) server_negotiation.cc:407] Connection header received
0621 06:58:32.038308 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.038315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.038394 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.038505 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.039915 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.040985 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.042926 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.043582 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.043697 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.048109 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.048115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.048121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.048182 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.054114 (+ 5932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.055367 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.055372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.055376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.055544 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.056050 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.056058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.056063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.056504 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.056651 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.057038 (+ 387us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.057279 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15858,"thread_start_us":16232,"threads_started":1}
I20260621 06:58:32.057770 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.022209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:47585)
0621 06:58:32.035001 (+ 12792us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.035024 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.035144 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.038762 (+ 3618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.038768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.039389 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.039717 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.039728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.041134 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.041138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.042746 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.042763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.043940 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.043950 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.044125 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.045314 (+ 1189us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.045342 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.047866 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.054253 (+ 6387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.054303 (+ 50us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.054311 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.055040 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.055660 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.055668 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.055675 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.055885 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.056616 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.056621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.056766 (+ 145us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.056903 (+ 137us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.057288 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12607,"thread_start_us":12471,"threads_started":1}
I20260621 06:58:32.058562 10283 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:32.058722 10283 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.059185 10283 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
I20260621 06:58:32.060168 9164 ts_manager.cc:194] Re-registered known tserver with Master: 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131)
I20260621 06:58:32.061002 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:47585
I20260621 06:58:32.061333 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.991533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.2.245.194:43569)
0621 06:58:32.021352 (+ 29819us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.021371 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.021517 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.029504 (+ 7987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.029510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.030167 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.030470 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.030479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.032398 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.032418 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.037701 (+ 5283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.037723 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.041363 (+ 3640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.041370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.041562 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.043020 (+ 1458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.043040 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.047175 (+ 4135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.057174 (+ 9999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.057182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.057188 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.057918 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.058526 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.058534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.058541 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.058734 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.059674 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.059682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.059999 (+ 317us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.060292 (+ 293us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.060706 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29681,"mutex_wait_us":810,"thread_start_us":28507,"threads_started":1}
I20260621 06:58:32.062780 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:31.990118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:43569 (local address 127.2.245.253:33397)
0621 06:58:32.019643 (+ 29525us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.019650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.021500 (+ 1850us) server_negotiation.cc:407] Connection header received
0621 06:58:32.021665 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.021673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.021781 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.021922 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.031008 (+ 9086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.032095 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.038675 (+ 6580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.039304 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.039457 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.049224 (+ 9767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.049233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.049241 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.049321 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.056974 (+ 7653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.058091 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.058123 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.058127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.058264 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.059002 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.059010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.059016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.059446 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.061764 (+ 2318us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.061957 (+ 193us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.062261 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29404,"thread_start_us":29303,"threads_started":1}
I20260621 06:58:32.063656 10284 heartbeater.cc:347] Connected to a master server at 127.2.245.253:33397
I20260621 06:58:32.063784 10284 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.064133 10284 heartbeater.cc:511] Master 127.2.245.253:33397 requested a full tablet report, sending...
I20260621 06:58:32.065199 9764 ts_manager.cc:194] Registered new tserver with Master: 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131)
I20260621 06:58:32.144783 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9492
I20260621 06:58:32.161748 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:33905
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:38303
--webserver_port=38211
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 06:58:32.451515 10138 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
W20260621 06:58:32.493511 10295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:32.493789 10295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:32.493904 10295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:32.501237 10295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:32.501351 10295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:32.501487 10295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:32.511335 10295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:33905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=38211
--tserver_master_addrs=127.2.245.254:38303,127.2.245.253:33397,127.2.245.252:39637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:32.512668 10295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:32.513968 10295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:32.527079 10300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:32.527020 10301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:32.527052 10303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:32.528191 10295 server_base.cc:1034] running on GCE node
I20260621 06:58:32.528738 10295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:32.529659 10295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:32.530891 10295 hybrid_clock.cc:648] HybridClock initialized: now 1782025112530854 us; error 55 us; skew 500 ppm
I20260621 06:58:32.531232 10295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025112, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:32.534235 10295 init.cc:377] Logged in from keytab as oryx/127.2.245.195@KRBTEST.COM (short username oryx)
I20260621 06:58:32.535763 10295 webserver.cc:460] Webserver started at http://127.2.245.195:38211/ using document root <none> and password file <none>
I20260621 06:58:32.536484 10295 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:32.536638 10295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:32.542438 10295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:32.545229 10310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:32.546516 10295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:32.546700 10295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "593bc2a97b0944ea9d60eb9161798972"
format_stamp: "Formatted at 2026-06-21 06:58:29 on dist-test-slave-9zdv"
server_key: "e02ca8f64645bad09e42ddf1a9507eae"
server_key_iv: "59ec0821f78a2aa1587c0d5c77507177"
server_key_version: "encryptionkey@0"
I20260621 06:58:32.547758 10295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:32.581425 10295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:32.586025 10295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:32.586303 10295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:32.587294 10295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:32.588620 10295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:32.588752 10295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:32.588888 10295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:32.588972 10295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:32.623406 10295 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:33905
I20260621 06:58:32.623494 10422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:33905 every 8 connection(s)
I20260621 06:58:32.624871 10295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:32.633064 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10295
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025112, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:32.655421 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.627868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:55983 (local address 127.2.245.254:38303)
0621 06:58:32.628308 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.628314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.629579 (+ 1265us) server_negotiation.cc:407] Connection header received
0621 06:58:32.632860 (+ 3281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.632869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.632991 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.633096 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.635820 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.637202 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.640303 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.641129 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.641243 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.647048 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.647058 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.647066 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.647120 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.651199 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.652240 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.652246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.652248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.652395 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.653020 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.653027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.653030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.653441 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.653579 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.654701 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.655100 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":992,"threads_started":1}
I20260621 06:58:32.658779 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.628569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:55983)
0621 06:58:32.629415 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.629460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.632453 (+ 2993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.633674 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.633688 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.634568 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.635548 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.635596 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.637464 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.637472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.640104 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.640124 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.641637 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.641645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.642037 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.642969 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.643020 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.646684 (+ 3664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.651417 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.651426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.651448 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.652093 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.652577 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.652581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.652585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.652862 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.653681 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.653688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.654179 (+ 491us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.655711 (+ 1532us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.656833 (+ 1122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":346,"threads_started":1}
I20260621 06:58:32.660768 10424 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38303
I20260621 06:58:32.661093 10424 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.662055 10424 heartbeater.cc:511] Master 127.2.245.254:38303 requested a full tablet report, sending...
I20260621 06:58:32.663998 9164 ts_manager.cc:194] Re-registered known tserver with Master: 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:32.665571 9164 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:55983
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025112, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.253@KRBTEST.COM
Jun 21 06:58:32 dist-test-slave-9zdv krb5kdc[9119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025112, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.252@KRBTEST.COM
I20260621 06:58:32.683734 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.628595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.2.245.195:40205)
0621 06:58:32.651703 (+ 23108us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.651721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.651831 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.654064 (+ 2233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.654069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.654804 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.655498 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.655513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.657174 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.657182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.660040 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.660061 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.661743 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.661752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.662011 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.666757 (+ 4746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.666788 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.670290 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.679122 (+ 8832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.679130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.679135 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.680115 (+ 980us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.680622 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.680626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.680630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.680785 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.681442 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.681449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.681622 (+ 173us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.682191 (+ 569us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.683295 (+ 1104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22963,"mutex_wait_us":1,"thread_start_us":24107,"threads_started":1}
I20260621 06:58:32.684511 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.656292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.2.245.254:38303)
0621 06:58:32.656435 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.656442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.661224 (+ 4782us) server_negotiation.cc:407] Connection header received
0621 06:58:32.661481 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.661487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.661570 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.661635 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.664506 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.665962 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.667879 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.668447 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.668548 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.673208 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.673218 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.673225 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.673275 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.677363 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.680013 (+ 2650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.680021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.680027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.680196 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.682660 (+ 2464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.682666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.682669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.683050 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.683304 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.683934 (+ 630us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.684173 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":50}
I20260621 06:58:32.685602 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.627932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:40205 (local address 127.2.245.253:33397)
0621 06:58:32.653747 (+ 25815us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.653753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.653769 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:32.653822 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.653826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.653896 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.654152 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.655686 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.656957 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.660256 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.661190 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.661385 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.674673 (+ 13288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.674682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.674689 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.674745 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.678971 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.680262 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.680267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.680277 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.680453 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.680915 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.680920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.680923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.681230 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.681367 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.684206 (+ 2839us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.684428 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25705,"thread_start_us":25579,"threads_started":1}
I20260621 06:58:32.686672 10425 heartbeater.cc:347] Connected to a master server at 127.2.245.253:33397
I20260621 06:58:32.687050 10425 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.687872 10425 heartbeater.cc:511] Master 127.2.245.253:33397 requested a full tablet report, sending...
I20260621 06:58:32.689265 9764 ts_manager.cc:194] Registered new tserver with Master: 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:32.693778 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.630349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:57059 (local address 127.2.245.252:39637)
0621 06:58:32.651127 (+ 20778us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.651133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.653240 (+ 2107us) server_negotiation.cc:407] Connection header received
0621 06:58:32.653358 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.653367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.653480 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.653600 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.670522 (+ 16922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.672345 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.678147 (+ 5802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.678922 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.679061 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.686506 (+ 7445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.686512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.686517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.686559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.690364 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.691239 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.691246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.691249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.691384 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.691814 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.691819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.691822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.692145 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.692250 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.693190 (+ 940us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.693398 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20634,"thread_start_us":20788,"threads_started":1}
I20260621 06:58:32.694730 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.629998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.2.245.195:57059)
0621 06:58:32.653096 (+ 23098us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.653114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.653245 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.653848 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.653856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.662980 (+ 9124us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:32.663293 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.663308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.672668 (+ 9360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.672677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.674369 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.674388 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.679378 (+ 4990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.679385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.679574 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.680396 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:32.680418 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:32.686274 (+ 5856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:32.690507 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.690513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.690517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.691101 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.691529 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:32.691534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:32.691538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:32.691700 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:32.692332 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:32.692337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:32.692886 (+ 549us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.694002 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.694333 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22982,"thread_start_us":22210,"threads_started":1}
I20260621 06:58:32.695638 10423 heartbeater.cc:347] Connected to a master server at 127.2.245.252:39637
I20260621 06:58:32.695770 10423 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:32.696089 10423 heartbeater.cc:511] Master 127.2.245.252:39637 requested a full tablet report, sending...
I20260621 06:58:32.697074 9958 ts_manager.cc:194] Registered new tserver with Master: 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:32.699028 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.656403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.2.245.253:33397)
0621 06:58:32.675836 (+ 19433us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.675844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.675865 (+ 21us) server_negotiation.cc:407] Connection header received
0621 06:58:32.675939 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.675945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.676034 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.676161 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.679259 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.680823 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.683021 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.683804 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.683933 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.689425 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.689432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.689437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.689504 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.694477 (+ 4973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.695801 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.695853 (+ 52us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.695857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.696045 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.696533 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.696538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.696541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.696854 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.697088 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.698298 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.698551 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19260,"thread_start_us":21162,"threads_started":1}
I20260621 06:58:32.690802 9164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:32.702401 9164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:32.705960 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.664118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.2.245.252:39637)
0621 06:58:32.690332 (+ 26214us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.690338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.690364 (+ 26us) server_negotiation.cc:407] Connection header received
0621 06:58:32.690414 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.690419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.690490 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.690614 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.692458 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.693881 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.696374 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.697039 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.697155 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.700143 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:32.700148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:32.700151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:32.700179 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:32.703394 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.704089 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.704095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.704098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.704222 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:32.704576 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:32.704581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:32.704584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:32.704916 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:32.705139 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.705398 (+ 259us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.705604 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26069,"thread_start_us":26651,"threads_started":1}
I20260621 06:58:32.746277 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.723960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:34131 (local address 127.0.0.1:34472)
0621 06:58:32.724520 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.724540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.724663 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.737846 (+ 13183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.737854 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.737892 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:32.738403 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.738418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.740431 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.740438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.742222 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.742236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.744778 (+ 2542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.744783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.745595 (+ 812us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.745631 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.745722 (+ 91us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.745861 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":1,"thread_start_us":683,"threads_started":1}
I20260621 06:58:32.747157 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.724500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.2.245.194:34131)
0621 06:58:32.736642 (+ 12142us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.736657 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.736698 (+ 41us) server_negotiation.cc:407] Connection header received
0621 06:58:32.736839 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.736852 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.737223 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.737501 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:32.739067 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.740293 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.742398 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.744285 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.744395 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.744617 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.746686 (+ 2069us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.746866 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11623,"thread_start_us":11896,"threads_started":1}
I20260621 06:58:32.749590 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.725971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:37096)
0621 06:58:32.741366 (+ 15395us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.741385 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.741500 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.742307 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.742313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.742341 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:32.742722 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.742734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.745027 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.745031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.747193 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.747209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.748770 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.748776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.748966 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.748996 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.749116 (+ 120us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.749257 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15272,"thread_start_us":15602,"threads_started":1}
I20260621 06:58:32.749646 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.726297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.2.245.193:43637)
0621 06:58:32.738992 (+ 12695us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.739000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.741517 (+ 2517us) server_negotiation.cc:407] Connection header received
0621 06:58:32.741638 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.741647 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.741916 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.742107 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:32.743380 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.744867 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.747375 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.748394 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.748474 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.748621 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.749164 (+ 543us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.749342 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12512,"thread_start_us":12402,"threads_started":1}
I20260621 06:58:32.752830 10217 tablet_service.cc:1467] Processing CreateTablet for tablet 9084902d7f104642b91a56efd48b0653 (DEFAULT_TABLE table=test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:32.753501 10072 tablet_service.cc:1467] Processing CreateTablet for tablet 9084902d7f104642b91a56efd48b0653 (DEFAULT_TABLE table=test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:32.754295 10217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9084902d7f104642b91a56efd48b0653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:32.754768 10072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9084902d7f104642b91a56efd48b0653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:32.757782 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.742282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52870 (local address 127.2.245.195:33905)
0621 06:58:32.751560 (+ 9278us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.751569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.751603 (+ 34us) server_negotiation.cc:407] Connection header received
0621 06:58:32.751702 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.751709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.751949 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.752167 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:32.752500 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.753638 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.755422 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.756802 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.756947 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.757152 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.757296 (+ 144us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.757448 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9088,"thread_start_us":8899,"threads_started":1}
I20260621 06:58:32.758208 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.748704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:33905 (local address 127.0.0.1:52870)
0621 06:58:32.748883 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.748903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.749004 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.752147 (+ 3143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.752151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.752177 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:32.752354 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.752363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.753768 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.753774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.755237 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.755253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.756953 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.756959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.757129 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.757163 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.757255 (+ 92us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.757398 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260621 06:58:32.761231 10358 tablet_service.cc:1467] Processing CreateTablet for tablet 9084902d7f104642b91a56efd48b0653 (DEFAULT_TABLE table=test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:32.762452 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9084902d7f104642b91a56efd48b0653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:32.771768 10455 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Bootstrap starting.
I20260621 06:58:32.775499 10455 tablet_bootstrap.cc:654] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:32.771144 10456 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Bootstrap starting.
I20260621 06:58:32.779145 10457 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Bootstrap starting.
I20260621 06:58:32.779991 10456 tablet_bootstrap.cc:654] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:32.779867 10455 log.cc:824] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:32.781164 10456 log.cc:824] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:32.782531 10457 tablet_bootstrap.cc:654] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:32.783226 10455 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: No bootstrap required, opened a new log
I20260621 06:58:32.783494 10455 ts_tablet_manager.cc:1397] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260621 06:58:32.783690 10457 log.cc:824] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:32.783808 10456 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: No bootstrap required, opened a new log
I20260621 06:58:32.784056 10456 ts_tablet_manager.cc:1397] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260621 06:58:32.791929 10456 raft_consensus.cc:348] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.792299 10456 raft_consensus.cc:374] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:32.792382 10456 raft_consensus.cc:729] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c782c8644c7440eb9a3befd68a4fc500, State: Initialized, Role: FOLLOWER
I20260621 06:58:32.792989 10456 consensus_queue.cc:260] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.793706 10457 tablet_bootstrap.cc:492] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: No bootstrap required, opened a new log
I20260621 06:58:32.794435 10457 ts_tablet_manager.cc:1397] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260621 06:58:32.794649 10456 ts_tablet_manager.cc:1428] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260621 06:58:32.799391 10455 raft_consensus.cc:348] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.799846 10455 raft_consensus.cc:374] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:32.799952 10455 raft_consensus.cc:729] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ea24b03a784601b4d2b9c9dca562db, State: Initialized, Role: FOLLOWER
I20260621 06:58:32.800541 10455 consensus_queue.cc:260] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.801548 10283 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
I20260621 06:58:32.803299 10455 ts_tablet_manager.cc:1428] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Time spent starting tablet: real 0.020s user 0.012s sys 0.003s
I20260621 06:58:32.804869 10457 raft_consensus.cc:348] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.805285 10457 raft_consensus.cc:374] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:32.805392 10457 raft_consensus.cc:729] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593bc2a97b0944ea9d60eb9161798972, State: Initialized, Role: FOLLOWER
I20260621 06:58:32.806037 10457 consensus_queue.cc:260] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.807301 10424 heartbeater.cc:503] Master 127.2.245.254:38303 was elected leader, sending a full tablet report...
I20260621 06:58:32.807760 10457 ts_tablet_manager.cc:1428] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260621 06:58:32.860232 10461 raft_consensus.cc:482] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:32.860562 10461 raft_consensus.cc:504] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.862318 10461 leader_election.cc:288] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131), 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:32.868618 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.862739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:53157 (local address 127.2.245.194:34131)
0621 06:58:32.863015 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.863023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.863441 (+ 418us) server_negotiation.cc:407] Connection header received
0621 06:58:32.863655 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.863662 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.863748 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.863836 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:32.864339 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.865100 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.866815 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.867703 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.867901 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.867955 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.868337 (+ 382us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.868428 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 06:58:32.869065 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.862628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:34131 (local address 127.2.245.193:53157)
0621 06:58:32.863344 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.863359 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.863513 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.863875 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.863880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.863911 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:32.864185 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.864193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.865232 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.865238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.866663 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.866674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.867972 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.867978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.868150 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.868185 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.868286 (+ 101us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.868416 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":364,"threads_started":1}
I20260621 06:58:32.869493 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.863418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:33905 (local address 127.2.245.193:56811)
0621 06:58:32.864007 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:32.864031 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:32.864165 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:32.864459 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:32.864463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:32.864482 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:32.864786 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.864794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.865792 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.865798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.867348 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:32.867359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.868861 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.868867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:32.868974 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.868994 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 06:58:32.869071 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 06:58:32.869197 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":327,"threads_started":1}
I20260621 06:58:32.869506 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.863553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:56811 (local address 127.2.245.195:33905)
0621 06:58:32.863787 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.863792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.864113 (+ 321us) server_negotiation.cc:407] Connection header received
0621 06:58:32.864298 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.864302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.864357 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.864447 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:32.864924 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.865662 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.867507 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.868570 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.868721 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.868805 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.869121 (+ 316us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.869261 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 06:58:32.870097 10237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9084902d7f104642b91a56efd48b0653" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea24b03a784601b4d2b9c9dca562db" is_pre_election: true
I20260621 06:58:32.870350 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9084902d7f104642b91a56efd48b0653" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593bc2a97b0944ea9d60eb9161798972" is_pre_election: true
I20260621 06:58:32.871111 10237 raft_consensus.cc:2454] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 0.
I20260621 06:58:32.871106 10378 raft_consensus.cc:2454] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 0.
I20260621 06:58:32.871754 10026 leader_election.cc:302] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 593bc2a97b0944ea9d60eb9161798972, c782c8644c7440eb9a3befd68a4fc500; no voters:
I20260621 06:58:32.872133 10461 raft_consensus.cc:2781] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:32.872315 10461 raft_consensus.cc:482] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:32.872403 10461 raft_consensus.cc:3037] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:32.874501 10461 raft_consensus.cc:504] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.875213 10461 leader_election.cc:288] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 election: Requested vote from peers 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131), 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:32.875571 10237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9084902d7f104642b91a56efd48b0653" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea24b03a784601b4d2b9c9dca562db"
I20260621 06:58:32.875622 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9084902d7f104642b91a56efd48b0653" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593bc2a97b0944ea9d60eb9161798972"
I20260621 06:58:32.875771 10237 raft_consensus.cc:3037] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:32.875839 10378 raft_consensus.cc:3037] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Advancing to term 1
W20260621 06:58:32.877913 10426 tablet.cc:2369] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:58:32.878610 10378 raft_consensus.cc:2454] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 1.
I20260621 06:58:32.878969 10026 leader_election.cc:302] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 593bc2a97b0944ea9d60eb9161798972, c782c8644c7440eb9a3befd68a4fc500; no voters:
I20260621 06:58:32.879256 10461 raft_consensus.cc:2781] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:32.879797 10461 raft_consensus.cc:686] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 1 LEADER]: Becoming Leader. State: Replica: c782c8644c7440eb9a3befd68a4fc500, State: Running, Role: LEADER
I20260621 06:58:32.879974 10237 raft_consensus.cc:2454] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 1.
I20260621 06:58:32.880389 10461 consensus_queue.cc:237] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } }
I20260621 06:58:32.885419 9164 catalog_manager.cc:5495] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 reported cstate change: term changed from 0 to 1, leader changed from <none> to c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "c782c8644c7440eb9a3befd68a4fc500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } health_report { overall_health: UNKNOWN } } }
W20260621 06:58:32.901283 10140 tablet.cc:2369] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:58:32.926553 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.921590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.2.245.193:43637)
0621 06:58:32.921750 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.921756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.921794 (+ 38us) server_negotiation.cc:407] Connection header received
0621 06:58:32.921907 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.921915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.922032 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.922194 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:32.922562 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.923407 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.924676 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:32.925063 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:32.925158 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:32.925577 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:32.926111 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:32.926218 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:32.926284 (+ 66us) server_negotiation.cc:299] Negotiation successful
0621 06:58:32.926348 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 06:58:32.935369 10378 raft_consensus.cc:1261] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Refusing update from remote peer c782c8644c7440eb9a3befd68a4fc500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:32.935484 10237 raft_consensus.cc:1261] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Refusing update from remote peer c782c8644c7440eb9a3befd68a4fc500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:32.936467 10466 consensus_queue.cc:1035] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:32.936867 10461 consensus_queue.cc:1035] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:32.950918 10469 mvcc.cc:205] Tried to move back new op lower bound from 7299174862572081152 to 7299174862361620480. Current Snapshot: MvccSnapshot[applied={T|T < 7299174862572081152}]
I20260621 06:58:32.953095 10471 mvcc.cc:205] Tried to move back new op lower bound from 7299174862572081152 to 7299174862361620480. Current Snapshot: MvccSnapshot[applied={T|T < 7299174862572081152}]
I20260621 06:58:32.972759 9163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:32.973062 9163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:32.979796 9163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb53aeb7e9f94cdaaff92e1a3acbfec2: Sending DeleteTablet for 3 replicas of tablet 9084902d7f104642b91a56efd48b0653
I20260621 06:58:32.980868 10217 tablet_service.cc:1514] Processing DeleteTablet for tablet 9084902d7f104642b91a56efd48b0653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:32 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:34472
I20260621 06:58:32.981083 10358 tablet_service.cc:1514] Processing DeleteTablet for tablet 9084902d7f104642b91a56efd48b0653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:32 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:52870
I20260621 06:58:32.981305 10072 tablet_service.cc:1514] Processing DeleteTablet for tablet 9084902d7f104642b91a56efd48b0653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:32 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37096
I20260621 06:58:32.992904 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9132
I20260621 06:58:32.993883 9194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:32.999356 10480 tablet_replica.cc:330] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: stopping tablet replica
I20260621 06:58:33.001935 10480 raft_consensus.cc:2229] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:33.005396 10481 tablet_replica.cc:330] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: stopping tablet replica
I20260621 06:58:33.005896 10481 raft_consensus.cc:2229] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:33.006413 10481 raft_consensus.cc:2258] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:33.008175 10480 raft_consensus.cc:2258] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:33.010473 10480 ts_tablet_manager.cc:1905] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:33.015547 10482 tablet_replica.cc:330] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: stopping tablet replica
I20260621 06:58:33.016074 10482 raft_consensus.cc:2229] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:33.016831 10482 raft_consensus.cc:2258] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:33.019233 10482 ts_tablet_manager.cc:1905] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:33.021896 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.986402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.2.245.253:33397)
0621 06:58:32.986636 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.986645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:32.990839 (+ 4194us) server_negotiation.cc:407] Connection header received
0621 06:58:32.992306 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:32.992315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:32.992423 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:32.992515 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:32.999775 (+ 7260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.001248 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.007101 (+ 5853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.007764 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.007904 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:33.011718 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:33.011726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:33.011731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:33.011776 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:33.015119 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.016005 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.016010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.016013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.016157 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.016606 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.016611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.016613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.017000 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:33.017236 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:33.021336 (+ 4100us) server_negotiation.cc:299] Negotiation successful
0621 06:58:33.021568 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 06:58:33.027431 10482 ts_tablet_manager.cc:1918] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:33.027891 10482 log.cc:1196] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/9084902d7f104642b91a56efd48b0653
I20260621 06:58:33.028601 10482 ts_tablet_manager.cc:1939] T 9084902d7f104642b91a56efd48b0653 P c782c8644c7440eb9a3befd68a4fc500: Deleting consensus metadata
I20260621 06:58:33.030702 10481 ts_tablet_manager.cc:1905] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:33.038282 10481 ts_tablet_manager.cc:1918] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:33.038476 10481 log.cc:1196] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/9084902d7f104642b91a56efd48b0653
I20260621 06:58:33.038990 10481 ts_tablet_manager.cc:1939] T 9084902d7f104642b91a56efd48b0653 P 593bc2a97b0944ea9d60eb9161798972: Deleting consensus metadata
I20260621 06:58:33.046617 10480 ts_tablet_manager.cc:1918] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:33.046864 10480 log.cc:1196] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/9084902d7f104642b91a56efd48b0653
I20260621 06:58:33.047521 10480 ts_tablet_manager.cc:1939] T 9084902d7f104642b91a56efd48b0653 P 10ea24b03a784601b4d2b9c9dca562db: Deleting consensus metadata
I20260621 06:58:33.050647 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:32.991048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.2.245.252:39637)
0621 06:58:32.991276 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:32.991285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:33.018428 (+ 27143us) server_negotiation.cc:407] Connection header received
0621 06:58:33.018707 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:33.018713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:33.018798 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:33.018880 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:33.022169 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.023286 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.025091 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.025716 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.025813 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:33.043217 (+ 17404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:33.043223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:33.043229 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:33.043274 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:33.047259 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.048275 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.048280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.048284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.048432 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.048888 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.048893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.048896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.049411 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:33.049575 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:33.050101 (+ 526us) server_negotiation.cc:299] Negotiation successful
0621 06:58:33.050320 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 06:58:33.185209 9149 catalog_manager.cc:4841] TS 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131): tablet 9084902d7f104642b91a56efd48b0653 (table test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]) successfully deleted
I20260621 06:58:33.186757 9149 catalog_manager.cc:4841] TS 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905): tablet 9084902d7f104642b91a56efd48b0653 (table test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]) successfully deleted
I20260621 06:58:33.187036 9149 catalog_manager.cc:4841] TS c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637): tablet 9084902d7f104642b91a56efd48b0653 (table test-table [id=efd4fb9b6e024c32a6e2f052f23dd3fb]) successfully deleted
I20260621 06:58:33.187781 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9132
I20260621 06:58:33.231009 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:33.213578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.2.245.253:33397)
0621 06:58:33.213816 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:33.213825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:33.214812 (+ 987us) server_negotiation.cc:407] Connection header received
0621 06:58:33.215878 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:33.215885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:33.215965 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:33.216063 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:33.218101 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.219132 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.220477 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.221076 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.221224 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:33.224776 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:33.224786 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:33.224794 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:33.224851 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:33.228156 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.228924 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.228930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.228933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.229053 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.229467 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.229472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.229475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.229816 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:33.230047 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:33.230389 (+ 342us) server_negotiation.cc:299] Negotiation successful
0621 06:58:33.230605 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260621 06:58:33.231212 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:33.214769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.2.245.252:39637)
0621 06:58:33.215138 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:33.215147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:33.215562 (+ 415us) server_negotiation.cc:407] Connection header received
0621 06:58:33.215878 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:33.215885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:33.215965 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:33.216040 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:33.217731 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.218759 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.220703 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:33.221335 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:33.221455 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:33.224226 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:33.224231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:33.224234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:33.224273 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:33.227723 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.228566 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.228574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.228580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.228727 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:33.229149 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:33.229155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:33.229158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:33.229477 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:33.229711 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:33.230601 (+ 890us) server_negotiation.cc:299] Negotiation successful
0621 06:58:33.230843 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 06:58:33.958518 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:33.957703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:34685)
0621 06:58:33.958206 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:33.958340 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":279,"threads_started":1}
W20260621 06:58:33.959002 10424 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:38303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:33.964568 10493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:33.963955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:44615)
0621 06:58:33.964322 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:33.964425 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":195,"threads_started":1}
W20260621 06:58:33.964920 10138 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:38303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:33.970676 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:33.969973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:51981)
0621 06:58:33.970396 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:33.970522 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":211,"threads_started":1}
W20260621 06:58:33.971177 10283 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:38303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.479928 10495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb53aeb7e9f94cdaaff92e1a3acbfec2)
I20260621 06:58:34.480235 10495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } }
I20260621 06:58:34.481624 10495 leader_election.cc:288] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303), 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253:33397)
I20260621 06:58:34.482831 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.481705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38248)
0621 06:58:34.482249 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.482540 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"spinlock_wait_cycles":5504,"thread_start_us":367,"threads_started":1}
W20260621 06:58:34.483152 9943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.484215 10500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb53aeb7e9f94cdaaff92e1a3acbfec2)
I20260621 06:58:34.484468 10500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } }
I20260621 06:58:34.484869 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.484464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38252)
0621 06:58:34.484614 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.484690 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 06:58:34.485018 9943 leader_election.cc:334] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303): Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.485920 10500 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303), 8add484e2fc04e7e8f57750de2239623 (127.2.245.252:39637)
I20260621 06:58:34.487016 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.485990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38260)
0621 06:58:34.486491 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.486686 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":299,"threads_started":1}
W20260621 06:58:34.488085 9749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.489616 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.481996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:33397 (local address 127.0.0.1:48744)
0621 06:58:34.482664 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.482683 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:34.482822 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:34.483367 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:34.483372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:34.483397 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:34.483815 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.483830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.485322 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.485326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.486709 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.486723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.488826 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.488832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.489025 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.489060 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 06:58:34.489134 (+ 74us) client_negotiation.cc:240] Negotiation successful
0621 06:58:34.489278 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":303,"threads_started":1}
I20260621 06:58:34.489941 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.489352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38276)
0621 06:58:34.489525 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.489720 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260621 06:58:34.490381 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.481969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.2.245.253:33397)
0621 06:58:34.482915 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.482923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.482944 (+ 21us) server_negotiation.cc:407] Connection header received
0621 06:58:34.483110 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.483113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.483176 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.483294 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:34.484046 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.485175 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.486867 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.488582 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.489557 (+ 975us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.489680 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.489737 (+ 57us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.489892 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":616,"threads_started":1}
W20260621 06:58:34.490296 9749 leader_election.cc:334] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303): Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.491005 9774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8add484e2fc04e7e8f57750de2239623" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0dc56c3e4389491d9d909f0662822aa8" is_pre_election: true
I20260621 06:58:34.491381 9774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8add484e2fc04e7e8f57750de2239623 in term 1.
I20260621 06:58:34.491882 9945 leader_election.cc:302] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc56c3e4389491d9d909f0662822aa8, 8add484e2fc04e7e8f57750de2239623; no voters: cb53aeb7e9f94cdaaff92e1a3acbfec2
I20260621 06:58:34.492252 10495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260621 06:58:34.492383 10495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb53aeb7e9f94cdaaff92e1a3acbfec2)
I20260621 06:58:34.492501 10495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 1 FOLLOWER]: Advancing to term 2
I20260621 06:58:34.495116 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.486702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39637 (local address 127.0.0.1:52104)
0621 06:58:34.487310 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.487333 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:34.487489 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:34.488768 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:34.488772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:34.488803 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:34.489040 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.489049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.490758 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.490763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.492765 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.492777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.494566 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.494570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.494679 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.494702 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 06:58:34.494770 (+ 68us) client_negotiation.cc:240] Negotiation successful
0621 06:58:34.494857 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":1091,"threads_started":1}
I20260621 06:58:34.495348 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.486851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.2.245.252:39637)
0621 06:58:34.488353 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.488359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.488381 (+ 22us) server_negotiation.cc:407] Connection header received
0621 06:58:34.488438 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.488446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.488543 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.489177 (+ 634us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:34.489419 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.490621 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.492909 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.494394 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.494515 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.494633 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.494823 (+ 190us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.494975 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1275,"threads_started":1}
I20260621 06:58:34.495836 9968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dc56c3e4389491d9d909f0662822aa8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8add484e2fc04e7e8f57750de2239623" is_pre_election: true
I20260621 06:58:34.495939 10495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } }
I20260621 06:58:34.496405 9968 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0dc56c3e4389491d9d909f0662822aa8 in current term 2: Already voted for candidate 8add484e2fc04e7e8f57750de2239623 in this term.
I20260621 06:58:34.496979 10495 leader_election.cc:288] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 election: Requested vote from peers cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303), 0dc56c3e4389491d9d909f0662822aa8 (127.2.245.253:33397)
I20260621 06:58:34.496958 9749 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0dc56c3e4389491d9d909f0662822aa8; no voters: 8add484e2fc04e7e8f57750de2239623, cb53aeb7e9f94cdaaff92e1a3acbfec2
I20260621 06:58:34.497128 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.496707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38280)
0621 06:58:34.496895 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.496992 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 06:58:34.497282 10500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 1 FOLLOWER]: Advancing to term 2
I20260621 06:58:34.497411 9774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8add484e2fc04e7e8f57750de2239623" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0dc56c3e4389491d9d909f0662822aa8"
I20260621 06:58:34.499500 10500 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 06:58:34.499841 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.499487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38294)
0621 06:58:34.499627 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.499712 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260621 06:58:34.500011 9943 leader_election.cc:334] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303): Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
I20260621 06:58:34.501700 9774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8add484e2fc04e7e8f57750de2239623 in term 2.
I20260621 06:58:34.502701 9945 leader_election.cc:302] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc56c3e4389491d9d909f0662822aa8, 8add484e2fc04e7e8f57750de2239623; no voters: cb53aeb7e9f94cdaaff92e1a3acbfec2
I20260621 06:58:34.503032 10495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 2 FOLLOWER]: Leader election won for term 2
I20260621 06:58:34.503449 10495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [term 2 LEADER]: Becoming Leader. State: Replica: 8add484e2fc04e7e8f57750de2239623, State: Running, Role: LEADER
I20260621 06:58:34.503815 10495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } }
I20260621 06:58:34.505506 10505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8add484e2fc04e7e8f57750de2239623. Latest consensus state: current_term: 2 leader_uuid: "8add484e2fc04e7e8f57750de2239623" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:34.505885 10505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:34.506494 10507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:34.511036 10507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:34.511965 10507 catalog_manager.cc:1259] Loaded cluster ID: e862c6e13ebc41fdb7b68be0dee74ebf
I20260621 06:58:34.512032 10507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:34.513043 10507 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:34.513792 10507 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Loaded TSK: 0
I20260621 06:58:34.514546 10507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:58:34.567045 9957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:34.574026 9957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:34.578066 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.577643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38332)
0621 06:58:34.577792 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.577890 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260621 06:58:34.578239 9774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [term 2 FOLLOWER]: Refusing update from remote peer 8add484e2fc04e7e8f57750de2239623: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260621 06:58:34.579064 10505 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260621 06:58:34.579224 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.578794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38346)
0621 06:58:34.579010 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.579077 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260621 06:58:34.579421 9943 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 -> Peer cb53aeb7e9f94cdaaff92e1a3acbfec2 (127.2.245.254:38303): Couldn't send request to peer cb53aeb7e9f94cdaaff92e1a3acbfec2. Status: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260621 06:58:34.581741 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8add484e2fc04e7e8f57750de2239623. Latest consensus state: current_term: 2 leader_uuid: "8add484e2fc04e7e8f57750de2239623" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:34.582021 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:34.583974 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8add484e2fc04e7e8f57750de2239623" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:34.584300 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc56c3e4389491d9d909f0662822aa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 06:58:34.587872 10513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8add484e2fc04e7e8f57750de2239623" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cb53aeb7e9f94cdaaff92e1a3acbfec2" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38303 } } peers { permanent_uuid: "0dc56c3e4389491d9d909f0662822aa8" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 33397 } attrs { promote: false } } peers { permanent_uuid: "8add484e2fc04e7e8f57750de2239623" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39637 } attrs { promote: false } } }
I20260621 06:58:34.588395 10513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:34.608848 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.595853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:33905 (local address 127.0.0.1:40846)
0621 06:58:34.596010 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.596029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:34.596152 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:34.603195 (+ 7043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:34.603199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:34.603225 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:34.603590 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.603598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.604843 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.604851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.606408 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.606421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.608114 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.608121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.608307 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.608341 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 06:58:34.608403 (+ 62us) client_negotiation.cc:240] Negotiation successful
0621 06:58:34.608536 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":33}
I20260621 06:58:34.609598 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.595357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:34131 (local address 127.0.0.1:53422)
0621 06:58:34.595560 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.595580 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:34.595694 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:34.603265 (+ 7571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:34.603270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:34.603299 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:34.603564 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.603581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.605817 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.605824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.607413 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.607424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.608908 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.608914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.609078 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.609100 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 06:58:34.609211 (+ 111us) client_negotiation.cc:240] Negotiation successful
0621 06:58:34.609335 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":32}
I20260621 06:58:34.609671 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.595481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.2.245.194:34131)
0621 06:58:34.602890 (+ 7409us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.602894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.602910 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:34.603013 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.603018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.603080 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.603251 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:34.604535 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.605663 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.607570 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.608759 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.608902 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.608995 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.609239 (+ 244us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.609370 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7307,"thread_start_us":7222,"threads_started":1}
I20260621 06:58:34.609778 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.596700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.2.245.195:33905)
0621 06:58:34.602753 (+ 6053us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.602763 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.602787 (+ 24us) server_negotiation.cc:407] Connection header received
0621 06:58:34.602879 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.602888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.603030 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.603180 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:34.603753 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.604723 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.606577 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.607870 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.609189 (+ 1319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.609296 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.609348 (+ 52us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.609481 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5893,"thread_start_us":5763,"threads_started":1}
I20260621 06:58:34.610539 10358 tablet_service.cc:1467] Processing CreateTablet for tablet 576df2547a2e446395e37a4f5752abbc (DEFAULT_TABLE table=test-table [id=538c51c0f7714580b4e1c2b041362c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:34.611055 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 576df2547a2e446395e37a4f5752abbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:34.611200 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.596213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52606)
0621 06:58:34.596755 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.596773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:34.596868 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:34.604156 (+ 7288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:34.604161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:34.604184 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:34.604489 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.604499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.605842 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.605848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.608136 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:34.608148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.610395 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.610400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:34.610566 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.610597 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 06:58:34.610723 (+ 126us) client_negotiation.cc:240] Negotiation successful
0621 06:58:34.610842 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":585,"threads_started":1}
I20260621 06:58:34.611347 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.596350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.2.245.193:43637)
0621 06:58:34.603826 (+ 7476us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.603833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.603854 (+ 21us) server_negotiation.cc:407] Connection header received
0621 06:58:34.603920 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.603926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.604015 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.604135 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:34.604656 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.605628 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.608301 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.610194 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.610317 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.610387 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.610764 (+ 377us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.610883 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7354,"thread_start_us":7101,"threads_started":1}
I20260621 06:58:34.612118 10217 tablet_service.cc:1467] Processing CreateTablet for tablet 576df2547a2e446395e37a4f5752abbc (DEFAULT_TABLE table=test-table [id=538c51c0f7714580b4e1c2b041362c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:34.612195 10072 tablet_service.cc:1467] Processing CreateTablet for tablet 576df2547a2e446395e37a4f5752abbc (DEFAULT_TABLE table=test-table [id=538c51c0f7714580b4e1c2b041362c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:34.612514 10217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 576df2547a2e446395e37a4f5752abbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:34.612597 10072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 576df2547a2e446395e37a4f5752abbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:34.618655 10518 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Bootstrap starting.
I20260621 06:58:34.619164 10520 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Bootstrap starting.
I20260621 06:58:34.619439 10519 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Bootstrap starting.
I20260621 06:58:34.620292 10518 tablet_bootstrap.cc:654] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:34.620993 10520 tablet_bootstrap.cc:654] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:34.621541 10519 tablet_bootstrap.cc:654] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:34.622475 10518 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: No bootstrap required, opened a new log
I20260621 06:58:34.622627 10518 ts_tablet_manager.cc:1397] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260621 06:58:34.623450 10518 raft_consensus.cc:348] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.623685 10518 raft_consensus.cc:374] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:34.623764 10518 raft_consensus.cc:729] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c782c8644c7440eb9a3befd68a4fc500, State: Initialized, Role: FOLLOWER
I20260621 06:58:34.623975 10518 consensus_queue.cc:260] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.627069 10519 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: No bootstrap required, opened a new log
I20260621 06:58:34.627239 10519 ts_tablet_manager.cc:1397] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260621 06:58:34.630103 10519 raft_consensus.cc:348] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.630450 10519 raft_consensus.cc:374] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:34.630563 10519 raft_consensus.cc:729] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593bc2a97b0944ea9d60eb9161798972, State: Initialized, Role: FOLLOWER
I20260621 06:58:34.630841 10518 ts_tablet_manager.cc:1428] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260621 06:58:34.630896 10519 consensus_queue.cc:260] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.635267 10520 tablet_bootstrap.cc:492] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: No bootstrap required, opened a new log
I20260621 06:58:34.635458 10520 ts_tablet_manager.cc:1397] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260621 06:58:34.636497 10520 raft_consensus.cc:348] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.636798 10520 raft_consensus.cc:374] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:34.636910 10520 raft_consensus.cc:729] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ea24b03a784601b4d2b9c9dca562db, State: Initialized, Role: FOLLOWER
I20260621 06:58:34.637174 10520 consensus_queue.cc:260] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.639869 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.632102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:58075)
0621 06:58:34.639424 (+ 7322us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.639584 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7198,"threads_started":1}
I20260621 06:58:34.640635 10519 ts_tablet_manager.cc:1428] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Time spent starting tablet: real 0.013s user 0.006s sys 0.003s
I20260621 06:58:34.641207 10137 heartbeater.cc:503] Master 127.2.245.252:39637 was elected leader, sending a full tablet report...
I20260621 06:58:34.641855 10520 ts_tablet_manager.cc:1428] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260621 06:58:34.642321 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.641827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:49713)
0621 06:58:34.642077 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.642179 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260621 06:58:34.643144 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.642132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:45743)
0621 06:58:34.642688 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.642867 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":481,"threads_started":1}
I20260621 06:58:34.644204 10423 heartbeater.cc:503] Master 127.2.245.252:39637 was elected leader, sending a full tablet report...
I20260621 06:58:34.651033 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.643746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:58961)
0621 06:58:34.650682 (+ 6936us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.650797 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6793,"threads_started":1}
I20260621 06:58:34.651791 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.645376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:38089)
0621 06:58:34.651437 (+ 6061us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.651585 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5896,"thread_start_us":5749,"threads_started":1}
I20260621 06:58:34.652063 10282 heartbeater.cc:503] Master 127.2.245.252:39637 was elected leader, sending a full tablet report...
I20260621 06:58:34.652782 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.652416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:51381)
0621 06:58:34.652580 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.652656 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 06:58:34.671306 10524 raft_consensus.cc:482] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:34.671558 10524 raft_consensus.cc:504] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.672221 10524 leader_election.cc:288] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131), 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:34.672748 10237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "576df2547a2e446395e37a4f5752abbc" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea24b03a784601b4d2b9c9dca562db" is_pre_election: true
I20260621 06:58:34.672803 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "576df2547a2e446395e37a4f5752abbc" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593bc2a97b0944ea9d60eb9161798972" is_pre_election: true
I20260621 06:58:34.672961 10237 raft_consensus.cc:2454] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 0.
I20260621 06:58:34.673090 10378 raft_consensus.cc:2454] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 0.
I20260621 06:58:34.673454 10026 leader_election.cc:302] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ea24b03a784601b4d2b9c9dca562db, c782c8644c7440eb9a3befd68a4fc500; no voters:
I20260621 06:58:34.673687 10524 raft_consensus.cc:2781] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:34.673787 10524 raft_consensus.cc:482] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:34.673892 10524 raft_consensus.cc:3037] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:34.675923 10524 raft_consensus.cc:504] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.676437 10524 leader_election.cc:288] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 election: Requested vote from peers 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131), 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905)
I20260621 06:58:34.676956 10237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "576df2547a2e446395e37a4f5752abbc" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea24b03a784601b4d2b9c9dca562db"
I20260621 06:58:34.676999 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "576df2547a2e446395e37a4f5752abbc" candidate_uuid: "c782c8644c7440eb9a3befd68a4fc500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "593bc2a97b0944ea9d60eb9161798972"
I20260621 06:58:34.677165 10237 raft_consensus.cc:3037] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:34.677189 10378 raft_consensus.cc:3037] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:34.679361 10237 raft_consensus.cc:2454] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 1.
I20260621 06:58:34.679363 10378 raft_consensus.cc:2454] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782c8644c7440eb9a3befd68a4fc500 in term 1.
I20260621 06:58:34.679761 10026 leader_election.cc:302] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ea24b03a784601b4d2b9c9dca562db, c782c8644c7440eb9a3befd68a4fc500; no voters:
I20260621 06:58:34.680048 10524 raft_consensus.cc:2781] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:34.680469 10524 raft_consensus.cc:686] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 1 LEADER]: Becoming Leader. State: Replica: c782c8644c7440eb9a3befd68a4fc500, State: Running, Role: LEADER
I20260621 06:58:34.680771 10524 consensus_queue.cc:237] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } }
I20260621 06:58:34.681579 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.681159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:48367)
0621 06:58:34.681303 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.681405 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 06:58:34.683686 9956 catalog_manager.cc:5495] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 reported cstate change: term changed from 0 to 1, leader changed from <none> to c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "c782c8644c7440eb9a3befd68a4fc500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c782c8644c7440eb9a3befd68a4fc500" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 43637 } health_report { overall_health: HEALTHY } } }
I20260621 06:58:34.707005 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.702919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.2.245.193:43637)
0621 06:58:34.703100 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.703105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.703121 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:34.703281 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.703284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.703345 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.703415 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:34.703746 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.704385 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.705492 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.705886 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.705971 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.706412 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:34.706512 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:34.706594 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.706711 (+ 117us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.706766 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 06:58:34.709872 10378 raft_consensus.cc:1261] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Refusing update from remote peer c782c8644c7440eb9a3befd68a4fc500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:34.709967 10237 raft_consensus.cc:1261] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Refusing update from remote peer c782c8644c7440eb9a3befd68a4fc500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:34.710675 10531 consensus_queue.cc:1035] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10ea24b03a784601b4d2b9c9dca562db" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:34.711021 10524 consensus_queue.cc:1035] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "593bc2a97b0944ea9d60eb9161798972" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 33905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:34.714150 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.713490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:49795)
0621 06:58:34.713653 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.713891 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260621 06:58:34.714246 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.713658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:56883)
0621 06:58:34.713869 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.714021 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260621 06:58:34.721477 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.721067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:41281)
0621 06:58:34.721208 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.721315 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260621 06:58:34.722713 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.722255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.193:58549)
0621 06:58:34.722465 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.722545 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260621 06:58:34.723374 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.722897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.195:59483)
0621 06:58:34.723118 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.723211 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260621 06:58:34.724076 10536 mvcc.cc:205] Tried to move back new op lower bound from 7299174869845815296 to 7299174869734678528. Current Snapshot: MvccSnapshot[applied={T|T < 7299174869845815296}]
I20260621 06:58:34.724591 9957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:34.724830 9957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:34.725176 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.724713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.2.245.194:60711)
0621 06:58:34.724936 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.725017 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260621 06:58:34.731582 9957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8add484e2fc04e7e8f57750de2239623: Sending DeleteTablet for 3 replicas of tablet 576df2547a2e446395e37a4f5752abbc
I20260621 06:58:34.732477 10217 tablet_service.cc:1514] Processing DeleteTablet for tablet 576df2547a2e446395e37a4f5752abbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:34 UTC) from {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:53422
I20260621 06:58:34.732729 10358 tablet_service.cc:1514] Processing DeleteTablet for tablet 576df2547a2e446395e37a4f5752abbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:34 UTC) from {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:40846
I20260621 06:58:34.732895 10357 tablet_service.cc:1514] Processing DeleteTablet for tablet 576df2547a2e446395e37a4f5752abbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:34 UTC) from {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:40846
I20260621 06:58:34.733490 10072 tablet_service.cc:1514] Processing DeleteTablet for tablet 576df2547a2e446395e37a4f5752abbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:34 UTC) from {username='oryx', principal='oryx/127.2.245.252@KRBTEST.COM'} at 127.0.0.1:52606
I20260621 06:58:34.733969 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9930
I20260621 06:58:34.743495 10544 tablet_replica.cc:330] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: stopping tablet replica
I20260621 06:58:34.743712 10544 raft_consensus.cc:2229] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:34.744005 10544 raft_consensus.cc:2258] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:34.744719 10543 tablet_replica.cc:330] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: stopping tablet replica
I20260621 06:58:34.744920 10543 raft_consensus.cc:2229] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:34.745149 10544 ts_tablet_manager.cc:1905] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:34.745198 10543 raft_consensus.cc:2258] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:34.746049 10543 ts_tablet_manager.cc:1905] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:34.748643 10545 tablet_replica.cc:330] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: stopping tablet replica
I20260621 06:58:34.748922 10545 raft_consensus.cc:2229] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:34.749375 10545 raft_consensus.cc:2258] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:34.750468 10545 ts_tablet_manager.cc:1905] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:34.752132 10544 ts_tablet_manager.cc:1918] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:34.752274 10544 log.cc:1196] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/576df2547a2e446395e37a4f5752abbc
I20260621 06:58:34.752400 10543 ts_tablet_manager.cc:1918] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:34.752521 10543 log.cc:1196] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/576df2547a2e446395e37a4f5752abbc
I20260621 06:58:34.752795 10544 ts_tablet_manager.cc:1939] T 576df2547a2e446395e37a4f5752abbc P 593bc2a97b0944ea9d60eb9161798972: Deleting consensus metadata
I20260621 06:58:34.752964 10543 ts_tablet_manager.cc:1939] T 576df2547a2e446395e37a4f5752abbc P 10ea24b03a784601b4d2b9c9dca562db: Deleting consensus metadata
I20260621 06:58:34.754068 9943 catalog_manager.cc:4841] TS 10ea24b03a784601b4d2b9c9dca562db (127.2.245.194:34131): tablet 576df2547a2e446395e37a4f5752abbc (table test-table [id=538c51c0f7714580b4e1c2b041362c4b]) successfully deleted
I20260621 06:58:34.754391 9943 catalog_manager.cc:4841] TS 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905): tablet 576df2547a2e446395e37a4f5752abbc (table test-table [id=538c51c0f7714580b4e1c2b041362c4b]) successfully deleted
W20260621 06:58:34.754915 9943 catalog_manager.cc:4808] TS 593bc2a97b0944ea9d60eb9161798972 (127.2.245.195:33905): delete failed for tablet 576df2547a2e446395e37a4f5752abbc because the tablet was not found. No further retry: Not found: Tablet not found: 576df2547a2e446395e37a4f5752abbc
I20260621 06:58:34.755529 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.735266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.2.245.193:43637)
0621 06:58:34.735483 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.735491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.737173 (+ 1682us) server_negotiation.cc:407] Connection header received
0621 06:58:34.737373 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.737380 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.737471 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.737599 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:34.739682 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.741209 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.743562 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.744309 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.744453 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.747684 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:34.747701 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:34.747721 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:34.747757 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:34.751872 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:34.752859 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:34.752868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:34.752871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:34.753048 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:34.753531 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:34.753536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:34.753539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:34.754000 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:34.754312 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.754893 (+ 581us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.755158 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 06:58:34.756291 10132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:34.871505 10545 ts_tablet_manager.cc:1918] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:34.871770 10545 log.cc:1196] T 576df2547a2e446395e37a4f5752abbc P c782c8644c7440eb9a3befd68a4fc500: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/576df2547a2e446395e37a4f5752abbc
I20260621 06:58:34.872154 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9930
I20260621 06:58:34.894646 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10149
I20260621 06:58:34.895617 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.895064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52640)
0621 06:58:34.895330 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.895432 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260621 06:58:34.895910 9943 catalog_manager.cc:4584] TS c782c8644c7440eb9a3befd68a4fc500 (127.2.245.193:43637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 576df2547a2e446395e37a4f5752abbc: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
I20260621 06:58:34.911736 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.895505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.2.245.194:34131)
0621 06:58:34.895735 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:34.895743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:34.895769 (+ 26us) server_negotiation.cc:407] Connection header received
0621 06:58:34.895942 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:34.895947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:34.896010 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:34.896139 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:34.897437 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.898124 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.899545 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:34.899953 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:34.900048 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:34.906519 (+ 6471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:34.906533 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:34.906574 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:34.906623 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:34.909348 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:34.909965 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:34.909971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:34.909973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:34.910085 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:34.910382 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:34.910385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:34.910387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:34.910724 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:34.910929 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:34.911245 (+ 316us) server_negotiation.cc:299] Negotiation successful
0621 06:58:34.911402 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 06:58:34.912446 10277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:34.956393 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.955838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52646)
0621 06:58:34.956065 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.956186 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260621 06:58:34.957466 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:34.957073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52652)
0621 06:58:34.957243 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:34.957323 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 06:58:35.006639 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10149
I20260621 06:58:35.023860 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10295
I20260621 06:58:35.025537 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.025155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52654)
0621 06:58:35.025324 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.025413 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260621 06:58:35.026495 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.026141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52666)
0621 06:58:35.026300 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.026379 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 06:58:35.037956 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.024665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.2.245.195:33905)
0621 06:58:35.024904 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:35.024910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:35.024929 (+ 19us) server_negotiation.cc:407] Connection header received
0621 06:58:35.025093 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:35.025097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:35.025158 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:35.025312 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:35.026870 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:35.027734 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:35.028918 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:35.029337 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:35.029481 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:35.032041 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:35.032055 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:35.032070 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:35.032102 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:35.035333 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:35.035955 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:35.035963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:35.035967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:35.036101 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:35.036495 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:35.036500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:35.036503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:35.036932 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:35.037202 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:35.037432 (+ 230us) server_negotiation.cc:299] Negotiation successful
0621 06:58:35.037647 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260621 06:58:35.038681 10418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:35.127628 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.127089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38354)
0621 06:58:35.127284 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.127398 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260621 06:58:35.128132 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.127486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52678)
0621 06:58:35.127866 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.127943 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":301,"threads_started":1}
I20260621 06:58:35.128757 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.128373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38303 (local address 127.0.0.1:38368)
0621 06:58:35.128521 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.128613 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:38303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260621 06:58:35.129153 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.128888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52688)
0621 06:58:35.128989 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.129055 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260621 06:58:35.131984 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10295
I20260621 06:58:35.151614 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9625
I20260621 06:58:35.152653 9794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:35.266615 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9625
I20260621 06:58:35.286609 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 9726
I20260621 06:58:35.287760 9988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:35.408422 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:35.407824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:43637 (local address 127.0.0.1:52692)
0621 06:58:35.408060 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:35.408162 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260621 06:58:35.408586 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 9726
2026-06-21T06:58:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9385 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:35 dist-test-slave-9zdv krb5kdc[10555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:35 dist-test-slave-9zdv krb5kdc[10555](info): set up 2 sockets
Jun 21 06:58:35 dist-test-slave-9zdv krb5kdc[10555](info): commencing operation
krb5kdc: starting...
W20260621 06:58:36.492249 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:36 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab.
Entry for principal oryx/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab.
I20260621 06:58:36.701889 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44133
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab
--principal=oryx/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:37.025285 10568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:37.025574 10568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:37.025755 10568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:37.033190 10568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:37.033320 10568 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 06:58:37.033391 10568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:37.033509 10568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:37.033589 10568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:37.033675 10568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:37.043658 10568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44133
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.254.keytab
--principal=oryx/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:37.044947 10568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:37.046288 10568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:37.058143 10574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:37.058162 10573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:37.058425 10576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:37.058976 10568 server_base.cc:1034] running on GCE node
I20260621 06:58:37.059755 10568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:37.061105 10568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:37.062350 10568 hybrid_clock.cc:648] HybridClock initialized: now 1782025117062293 us; error 35 us; skew 500 ppm
I20260621 06:58:37.062685 10568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:37 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025117, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:37.065742 10568 init.cc:377] Logged in from keytab as oryx/127.2.245.254@KRBTEST.COM (short username oryx)
I20260621 06:58:37.067258 10568 webserver.cc:460] Webserver started at http://127.2.245.254:37329/ using document root <none> and password file <none>
I20260621 06:58:37.067926 10568 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:37.068053 10568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:37.068360 10568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:37.071336 10568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "a58ace09664d47e2b8fb2eb8424fa8e5"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "9b9e0c9fc63f8855f455fd4857153a23"
server_key_iv: "951b52a350bfd5e41471a41476e9df2b"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.072088 10568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "a58ace09664d47e2b8fb2eb8424fa8e5"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "9b9e0c9fc63f8855f455fd4857153a23"
server_key_iv: "951b52a350bfd5e41471a41476e9df2b"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.076964 10568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:37.080211 10583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:37.081566 10568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:37.081749 10568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "a58ace09664d47e2b8fb2eb8424fa8e5"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "9b9e0c9fc63f8855f455fd4857153a23"
server_key_iv: "951b52a350bfd5e41471a41476e9df2b"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.081971 10568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:37.107990 10568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:37.111953 10568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:37.112221 10568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:37.133349 10568 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:44133
I20260621 06:58:37.133368 10644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:44133 every 8 connection(s)
I20260621 06:58:37.134621 10568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:37.138262 10645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:37.141842 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10568
I20260621 06:58:37.142045 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:37.142666 3031 external_mini_cluster.cc:1428] Setting key b1b426b5ec15a27fde7fd7627d3f1009
I20260621 06:58:37.150553 10645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Bootstrap starting.
I20260621 06:58:37.154165 10645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:37.155447 10645 log.cc:824] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:37.158429 10645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: No bootstrap required, opened a new log
Jun 21 06:58:37 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:37.167572 10645 raft_consensus.cc:348] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } }
I20260621 06:58:37.167927 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:37.144699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43580 (local address 127.2.245.254:44133)
0621 06:58:37.145554 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:37.145570 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:37.145621 (+ 51us) server_negotiation.cc:407] Connection header received
0621 06:58:37.148721 (+ 3100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:37.148750 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:37.149262 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:37.149698 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:37.151366 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.153254 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.154591 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.155693 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.155884 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:37.160436 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:37.160447 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:37.160462 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:37.160506 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:37.163757 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.164600 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.164607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.164613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.164782 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.165160 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.165163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.165165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.165689 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:37.166010 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:37.166400 (+ 390us) server_negotiation.cc:299] Negotiation successful
0621 06:58:37.166790 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":373,"threads_started":1}
I20260621 06:58:37.167992 10645 raft_consensus.cc:374] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:37.168100 10645 raft_consensus.cc:729] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58ace09664d47e2b8fb2eb8424fa8e5, State: Initialized, Role: FOLLOWER
I20260621 06:58:37.168637 10645 consensus_queue.cc:260] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } }
I20260621 06:58:37.168833 10645 raft_consensus.cc:388] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:37.168928 10645 raft_consensus.cc:482] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:37.169050 10645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:37.171535 10645 raft_consensus.cc:504] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } }
I20260621 06:58:37.172144 10645 leader_election.cc:302] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a58ace09664d47e2b8fb2eb8424fa8e5; no voters:
I20260621 06:58:37.172835 10645 leader_election.cc:288] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:37.172956 10650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:37.173668 10650 raft_consensus.cc:686] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [term 1 LEADER]: Becoming Leader. State: Replica: a58ace09664d47e2b8fb2eb8424fa8e5, State: Running, Role: LEADER
I20260621 06:58:37.174209 10650 consensus_queue.cc:237] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } }
I20260621 06:58:37.175104 10645 sys_catalog.cc:564] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:37.177327 10652 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a58ace09664d47e2b8fb2eb8424fa8e5. Latest consensus state: current_term: 1 leader_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } } }
I20260621 06:58:37.177219 10651 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58ace09664d47e2b8fb2eb8424fa8e5" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44133 } } }
I20260621 06:58:37.177728 10652 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:37.177728 10651 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5 [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:37.179448 10657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:37.184787 10657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:37.193706 10657 catalog_manager.cc:1347] Generated new cluster ID: b83ad05e4795455fa5a32865ae2f1767
I20260621 06:58:37.193822 10657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:37.210608 10657 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:37.212569 10657 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:37.224278 10657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Generated new TSK 0
I20260621 06:58:37.225164 10657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
I20260621 06:58:37.284492 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44133
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:37.643184 10671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:37.643460 10671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:37.643630 10671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:37.650691 10671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:37.650825 10671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:37.650997 10671 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:58:37.651126 10671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:37.661118 10671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:44133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:37.662427 10671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:37.663851 10671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:37.676548 10677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:37.676771 10679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:37.677013 10676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:37.677393 10671 server_base.cc:1034] running on GCE node
I20260621 06:58:37.678002 10671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:37.678895 10671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:37.680159 10671 hybrid_clock.cc:648] HybridClock initialized: now 1782025117680102 us; error 38 us; skew 500 ppm
I20260621 06:58:37.680488 10671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:37 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025117, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:37.683636 10671 init.cc:377] Logged in from keytab as oryx/127.2.245.193@KRBTEST.COM (short username oryx)
I20260621 06:58:37.685050 10671 webserver.cc:460] Webserver started at http://127.2.245.193:40041/ using document root <none> and password file <none>
I20260621 06:58:37.685743 10671 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:37.685863 10671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:37.686169 10671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:37.689216 10671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "3a183474e7484da3b52dbedbd879447f"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "e315601a5301c3018f9e84fe0798f16a"
server_key_iv: "5a5d3db2dd89fcf4f5cef2518245b1bf"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.690007 10671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "3a183474e7484da3b52dbedbd879447f"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "e315601a5301c3018f9e84fe0798f16a"
server_key_iv: "5a5d3db2dd89fcf4f5cef2518245b1bf"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.696332 10671 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260621 06:58:37.700107 10686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:37.701370 10671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:37.701561 10671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "3a183474e7484da3b52dbedbd879447f"
format_stamp: "Formatted at 2026-06-21 06:58:37 on dist-test-slave-9zdv"
server_key: "e315601a5301c3018f9e84fe0798f16a"
server_key_iv: "5a5d3db2dd89fcf4f5cef2518245b1bf"
server_key_version: "encryptionkey@0"
I20260621 06:58:37.701779 10671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:37.726425 10671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:37.731070 10671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:37.731344 10671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:37.734009 10671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:37.734115 10671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:37.734246 10671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:37.734309 10671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:58:37 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025117, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:37.759214 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:37.734796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:49625 (local address 127.2.245.254:44133)
0621 06:58:37.735283 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:37.735287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:37.736562 (+ 1275us) server_negotiation.cc:407] Connection header received
0621 06:58:37.738603 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:37.738608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:37.738695 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:37.738820 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:37.741014 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.742049 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.744749 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.745176 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.745303 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:37.750287 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:37.750301 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:37.750310 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:37.750360 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:37.754516 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.755682 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.755687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.755691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.755854 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.756584 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.756591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.756594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.757034 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:37.757204 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:37.758323 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0621 06:58:37.758652 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":295,"threads_started":1}
I20260621 06:58:37.760017 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:37.735586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.193:49625)
0621 06:58:37.736405 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:37.736441 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:37.738321 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:37.739164 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:37.739181 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:37.740002 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:37.740829 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:37.740848 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.742209 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.742213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:37.744573 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:37.744585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.745427 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.745434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:37.745699 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:37.746598 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:37.746636 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:37.749967 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:37.754715 (+ 4748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:37.754727 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:37.754743 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:37.755543 (+ 800us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:37.756019 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:37.756025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:37.756031 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:37.756393 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:37.757189 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:37.757203 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:37.757730 (+ 527us) client_negotiation.cc:769] Sending connection context
0621 06:58:37.758143 (+ 413us) client_negotiation.cc:240] Negotiation successful
0621 06:58:37.758647 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":365,"threads_started":1}
I20260621 06:58:37.779237 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:37.765567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.193:41555)
0621 06:58:37.766145 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:37.766161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:37.766303 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:37.766613 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:37.766618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:37.767145 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:37.767523 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:37.767533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.768812 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.768815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:37.770519 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:37.770535 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.771488 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.771495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:37.771631 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:37.772240 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:37.772268 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:37.772938 (+ 670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:37.776393 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:37.776399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:37.776403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:37.776833 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:37.777227 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:37.777232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:37.777235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:37.777360 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:37.777984 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:37.777989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:37.778210 (+ 221us) client_negotiation.cc:769] Sending connection context
0621 06:58:37.778383 (+ 173us) client_negotiation.cc:240] Negotiation successful
0621 06:58:37.778770 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":312,"threads_started":1}
I20260621 06:58:37.779675 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:37.765467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41555 (local address 127.2.245.254:44133)
0621 06:58:37.765605 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:37.765609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:37.766241 (+ 632us) server_negotiation.cc:407] Connection header received
0621 06:58:37.766415 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:37.766419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:37.766495 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:37.766593 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:37.767656 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.768609 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.770669 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:37.771281 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:37.771410 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:37.773117 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:37.773123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:37.773131 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:37.773159 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:37.776275 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.776951 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.776964 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.776967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.777110 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:37.777472 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:37.777478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:37.777484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:37.777841 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:37.778037 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:37.778838 (+ 801us) server_negotiation.cc:299] Negotiation successful
0621 06:58:37.779147 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 06:58:37.787079 10671 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:40545
I20260621 06:58:37.787102 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:40545 every 8 connection(s)
I20260621 06:58:37.788260 10671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:37.789356 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10671
I20260621 06:58:37.789562 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:37.790166 3031 external_mini_cluster.cc:1428] Setting key c93f4a30792be92ba5b4aed42db2db40
I20260621 06:58:37.790398 10807 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44133
I20260621 06:58:37.790625 10807 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:37.791419 10807 heartbeater.cc:511] Master 127.2.245.254:44133 requested a full tablet report, sending...
I20260621 06:58:37.793376 10600 ts_manager.cc:194] Registered new tserver with Master: 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545)
I20260621 06:58:37.794834 10600 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:49625
WARNING: no policy specified for oryx/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
I20260621 06:58:37.833899 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44133
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:38.157820 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:38.158118 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:38.158218 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:38.165673 10811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:38.165788 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:38.165875 10811 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:58:38.165956 10811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:38.176026 10811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:44133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:38.177336 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:38.178596 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:38.191848 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:38.191947 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:38.191947 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:38.192760 10811 server_base.cc:1034] running on GCE node
I20260621 06:58:38.193349 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:38.194247 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:38.195461 10811 hybrid_clock.cc:648] HybridClock initialized: now 1782025118195389 us; error 38 us; skew 500 ppm
I20260621 06:58:38.195791 10811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:38.199061 10811 init.cc:377] Logged in from keytab as oryx/127.2.245.194@KRBTEST.COM (short username oryx)
I20260621 06:58:38.200476 10811 webserver.cc:460] Webserver started at http://127.2.245.194:33289/ using document root <none> and password file <none>
I20260621 06:58:38.201160 10811 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:38.201289 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:38.201599 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:38.204535 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "ac9c56604d5f059251f5c41cb588a2e9"
server_key_iv: "c7bceaa040d3d56e488ca22b23632090"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.205375 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "ac9c56604d5f059251f5c41cb588a2e9"
server_key_iv: "c7bceaa040d3d56e488ca22b23632090"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.210407 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:58:38.213835 10826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:38.215238 10811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:38.215397 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "ac9c56604d5f059251f5c41cb588a2e9"
server_key_iv: "c7bceaa040d3d56e488ca22b23632090"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.215620 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:38.228720 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:38.232935 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:38.233217 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:38.235919 10811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:38.236034 10811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:38.236195 10811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:38.236266 10811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:38.262619 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.237044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.194:40731)
0621 06:58:38.237869 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:38.237907 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:38.239958 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:38.240782 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:38.240800 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:38.241685 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:38.242694 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.242745 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.244259 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.244265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.247199 (+ 2934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.247276 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.248632 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.248642 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.248963 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.249922 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:38.249957 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:38.254114 (+ 4157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:38.258199 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.258217 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.258243 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.258995 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.259503 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.259510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.259514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.259771 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.260333 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:38.260342 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:38.260706 (+ 364us) client_negotiation.cc:769] Sending connection context
0621 06:58:38.261024 (+ 318us) client_negotiation.cc:240] Negotiation successful
0621 06:58:38.261439 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":395,"threads_started":1}
I20260621 06:58:38.263641 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.236324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:40731 (local address 127.2.245.254:44133)
0621 06:58:38.236529 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:38.236537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:38.238034 (+ 1497us) server_negotiation.cc:407] Connection header received
0621 06:58:38.240278 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:38.240284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:38.240380 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:38.240489 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:38.243047 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.244070 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.247554 (+ 3484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.248144 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.248290 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.254559 (+ 6269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:38.254567 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:38.254572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:38.254607 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:38.257988 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.259166 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.259170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.259172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.259327 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.259906 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.259911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.259914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.260198 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:38.260363 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:38.263250 (+ 2887us) server_negotiation.cc:299] Negotiation successful
0621 06:58:38.263415 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 06:58:38.294205 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.277605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:33813 (local address 127.2.245.254:44133)
0621 06:58:38.277806 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:38.277815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:38.278541 (+ 726us) server_negotiation.cc:407] Connection header received
0621 06:58:38.278781 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:38.278787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:38.278893 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:38.279121 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:38.280636 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.281387 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.283915 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.284591 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.284762 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.288029 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:38.288038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:38.288045 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:38.288089 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:38.291477 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.292159 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.292166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.292169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.292337 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.292797 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.292800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.292802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.293116 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:38.293231 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:38.293811 (+ 580us) server_negotiation.cc:299] Negotiation successful
0621 06:58:38.293977 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 06:58:38.294871 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.277716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.194:33813)
0621 06:58:38.278406 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:38.278422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:38.278618 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:38.279174 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:38.279186 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:38.279963 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:38.280462 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.280478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.281694 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.281702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.283772 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.283790 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.284902 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.284913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.285230 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.286157 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:38.286185 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:38.287776 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:38.291598 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.291602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.291605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.292006 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.292462 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.292469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.292475 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.292678 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.293230 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:38.293237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:38.293525 (+ 288us) client_negotiation.cc:769] Sending connection context
0621 06:58:38.293730 (+ 205us) client_negotiation.cc:240] Negotiation successful
0621 06:58:38.294205 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":436,"threads_started":1}
I20260621 06:58:38.295732 10811 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:39935
I20260621 06:58:38.295755 10945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:39935 every 8 connection(s)
I20260621 06:58:38.297322 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:38.299705 10946 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44133
I20260621 06:58:38.299890 10946 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:38.300482 10946 heartbeater.cc:511] Master 127.2.245.254:44133 requested a full tablet report, sending...
I20260621 06:58:38.301633 10600 ts_manager.cc:194] Registered new tserver with Master: 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935)
I20260621 06:58:38.302503 10600 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:40731
I20260621 06:58:38.304147 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10811
I20260621 06:58:38.304316 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:38.304745 3031 external_mini_cluster.cc:1428] Setting key 86b67c4a67752fb87bdfee369fa288c3
WARNING: no policy specified for oryx/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
I20260621 06:58:38.350167 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44133
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:38.678395 10950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:38.678666 10950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:38.678761 10950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:38.686305 10950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:38.686414 10950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:38.686496 10950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 06:58:38.686582 10950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:38.696262 10950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.245.254:44133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:38.697503 10950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:38.698779 10950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:38.711776 10955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:38.711838 10956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:38.711812 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:38.712486 10950 server_base.cc:1034] running on GCE node
I20260621 06:58:38.713091 10950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:38.713974 10950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:38.715209 10950 hybrid_clock.cc:648] HybridClock initialized: now 1782025118715140 us; error 48 us; skew 500 ppm
I20260621 06:58:38.715517 10950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:38.718562 10950 init.cc:377] Logged in from keytab as oryx/127.2.245.195@KRBTEST.COM (short username oryx)
I20260621 06:58:38.720196 10950 webserver.cc:460] Webserver started at http://127.2.245.195:38621/ using document root <none> and password file <none>
I20260621 06:58:38.720840 10950 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:38.720922 10950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:38.721184 10950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:38.724056 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "97bcee678e0b4e0eb64532417dd4319e"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "6c140fa787bfade56b23ab05fb6a7403"
server_key_iv: "129f39fcab8a64962b9907a7b3c9b97b"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.724771 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "97bcee678e0b4e0eb64532417dd4319e"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "6c140fa787bfade56b23ab05fb6a7403"
server_key_iv: "129f39fcab8a64962b9907a7b3c9b97b"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.729732 10950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:58:38.733332 10965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:38.734710 10950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:58:38.734906 10950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "97bcee678e0b4e0eb64532417dd4319e"
format_stamp: "Formatted at 2026-06-21 06:58:38 on dist-test-slave-9zdv"
server_key: "6c140fa787bfade56b23ab05fb6a7403"
server_key_iv: "129f39fcab8a64962b9907a7b3c9b97b"
server_key_version: "encryptionkey@0"
I20260621 06:58:38.735173 10950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:38.757279 10950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:38.761231 10950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:38.761492 10950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:38.764389 10950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:38.764501 10950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:38.764657 10950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:38.764717 10950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:38.788193 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.764657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:45969 (local address 127.2.245.254:44133)
0621 06:58:38.764871 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:38.764877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:38.766402 (+ 1525us) server_negotiation.cc:407] Connection header received
0621 06:58:38.768497 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:38.768503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:38.768600 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:38.768741 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:38.771173 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.772258 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.774906 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.775383 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.775542 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.781293 (+ 5751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:38.781298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:38.781302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:38.781335 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:38.784360 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.785265 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.785269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.785271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.785391 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.785806 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.785812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.785817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.786182 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:38.786743 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:38.787614 (+ 871us) server_negotiation.cc:299] Negotiation successful
0621 06:58:38.787837 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 06:58:38.789251 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.765263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.195:45969)
0621 06:58:38.766222 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:38.766279 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:38.768225 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:38.769036 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:38.769051 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:38.769882 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:38.770980 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.771009 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.772424 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.772431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.774709 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.774720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.775658 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.775666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.775982 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.776911 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:38.776953 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:38.781029 (+ 4076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:38.784553 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.784565 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.784587 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.785090 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.785502 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.785508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.785512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.785688 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.786714 (+ 1026us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:38.786722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:38.787188 (+ 466us) client_negotiation.cc:769] Sending connection context
0621 06:58:38.787490 (+ 302us) client_negotiation.cc:240] Negotiation successful
0621 06:58:38.787999 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":483,"threads_started":1}
I20260621 06:58:38.798265 10807 heartbeater.cc:503] Master 127.2.245.254:44133 was elected leader, sending a full tablet report...
I20260621 06:58:38.817284 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.798357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:43623 (local address 127.2.245.254:44133)
0621 06:58:38.798533 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:38.798540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:38.799955 (+ 1415us) server_negotiation.cc:407] Connection header received
0621 06:58:38.800198 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:38.800206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:38.800303 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:38.800397 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:38.802200 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.803418 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.806281 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.807028 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.807181 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.809850 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:38.809854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:38.809857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:38.809886 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:38.813863 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.814890 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.814897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.814903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.815167 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:38.815685 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:38.815689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:38.815691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:38.815953 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:38.816052 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:38.816594 (+ 542us) server_negotiation.cc:299] Negotiation successful
0621 06:58:38.816854 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 06:58:38.817589 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:38.799051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.2.245.195:43623)
0621 06:58:38.799861 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:38.799880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:38.800061 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:38.800615 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:38.800622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:38.801414 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:38.801980 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.802000 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.803604 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.803611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.806034 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:38.806059 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.807266 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.807275 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:38.807516 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:38.808367 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:38.808398 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:38.809679 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:38.814018 (+ 4339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.814023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.814028 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.814739 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.815319 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:38.815330 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:38.815336 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:38.815580 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:38.816079 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:38.816084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:38.816314 (+ 230us) client_negotiation.cc:769] Sending connection context
0621 06:58:38.816493 (+ 179us) client_negotiation.cc:240] Negotiation successful
0621 06:58:38.816992 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":793,"threads_started":1}
I20260621 06:58:38.817899 10950 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:46051
I20260621 06:58:38.817924 11084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:46051 every 8 connection(s)
I20260621 06:58:38.819304 10950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:38.821592 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 10950
I20260621 06:58:38.821770 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:38.821741 11085 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44133
I20260621 06:58:38.821969 11085 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:38.822242 3031 external_mini_cluster.cc:1428] Setting key 463e258dad9587cf4109812fd1405e29
I20260621 06:58:38.822583 11085 heartbeater.cc:511] Master 127.2.245.254:44133 requested a full tablet report, sending...
I20260621 06:58:38.823930 10600 ts_manager.cc:194] Registered new tserver with Master: 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051)
I20260621 06:58:38.825008 10600 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:45969
I20260621 06:58:38.825381 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:38 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:39.305023 10946 heartbeater.cc:503] Master 127.2.245.254:44133 was elected leader, sending a full tablet report...
I20260621 06:58:39.827279 11085 heartbeater.cc:503] Master 127.2.245.254:44133 was elected leader, sending a full tablet report...
W20260621 06:58:39.946125 11093 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.075s user 0.004s sys 0.011s
W20260621 06:58:39.947499 10695 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:38.864501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.2.245.254:44133)
0621 06:58:38.864756 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:38.864765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:38.865366 (+ 601us) server_negotiation.cc:407] Connection header received
0621 06:58:38.865542 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:38.865550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:38.865652 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:38.865770 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:38.867665 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.868497 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.869743 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:38.870147 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:38.870259 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:39.947250 (+1076991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43584: BlockingRecv error: recv got EOF from 127.0.0.1:43584 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 21 06:58:39 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025118, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:39.972991 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:39.952719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.2.245.254:44133)
0621 06:58:39.952955 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:39.952970 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:39.953200 (+ 230us) server_negotiation.cc:407] Connection header received
0621 06:58:39.953386 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:39.953390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:39.953483 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:39.953566 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:39.954891 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:39.955677 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:39.956900 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:39.957305 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:39.957398 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:39.967934 (+ 10536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:39.967940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:39.967943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:39.967984 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:39.970610 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:39.971358 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:39.971362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:39.971364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:39.971487 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:39.971808 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:39.971811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:39.971813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:39.972079 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:39.972284 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:39.972512 (+ 228us) server_negotiation.cc:299] Negotiation successful
0621 06:58:39.972699 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 06:58:39.977365 10600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:39.984375 10600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:40.018019 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:39.996843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.2.245.195:46051)
0621 06:58:40.009333 (+ 12490us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.009346 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.009387 (+ 41us) server_negotiation.cc:407] Connection header received
0621 06:58:40.009525 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.009536 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.009852 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.010086 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.011312 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.012910 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.015109 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.016553 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.016693 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.016927 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.017538 (+ 611us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.017702 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12283,"thread_start_us":12135,"threads_started":1}
I20260621 06:58:40.018069 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:39.997400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.2.245.194:39935)
0621 06:58:40.007442 (+ 10042us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.007454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.009470 (+ 2016us) server_negotiation.cc:407] Connection header received
0621 06:58:40.009577 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.009587 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.009853 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.010054 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.011276 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.012456 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.015415 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.016855 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.016997 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.017219 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.017636 (+ 417us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.017799 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9835,"thread_start_us":10057,"threads_started":1}
I20260621 06:58:40.018110 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:39.997055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:39935 (local address 127.0.0.1:47268)
0621 06:58:40.009178 (+ 12123us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.009198 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.009335 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.010350 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.010358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.010399 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.011099 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.011113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.012758 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.012763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.015252 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.015267 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.017108 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.017115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.017313 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.017348 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.017448 (+ 100us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.017702 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11976,"mutex_wait_us":101,"spinlock_wait_cycles":21248,"thread_start_us":10780,"threads_started":1}
I20260621 06:58:40.019377 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.000088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:40545 (local address 127.0.0.1:46694)
0621 06:58:40.010421 (+ 10333us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.010439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.010601 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.011492 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.011496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.011521 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.011871 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.011882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.013674 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.013680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.015955 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.015972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.018535 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.018540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.018733 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.018798 (+ 65us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.018889 (+ 91us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.019037 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10212,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":10061,"threads_started":1}
I20260621 06:58:40.019712 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.000786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.2.245.193:40545)
0621 06:58:40.005761 (+ 4975us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.005775 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.010581 (+ 4806us) server_negotiation.cc:407] Connection header received
0621 06:58:40.010802 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.010812 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.011072 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.011262 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.012065 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.013517 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.016141 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.018039 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.018191 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.018397 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.019009 (+ 612us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.019212 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4756,"thread_start_us":4571,"threads_started":1}
I20260621 06:58:40.019785 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:39.996652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46051 (local address 127.0.0.1:32800)
0621 06:58:40.008361 (+ 11709us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.008423 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.008660 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.010355 (+ 1695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.010360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.010418 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.011073 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.011090 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.013096 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.013102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.014878 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.014891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.017063 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.017069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.017289 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.017338 (+ 49us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.017469 (+ 131us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.019407 (+ 1938us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11362,"mutex_wait_us":797,"spinlock_wait_cycles":8704,"thread_start_us":12220,"threads_started":1}
I20260621 06:58:40.022284 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 (DEFAULT_TABLE table=test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.022574 10737 tablet_service.cc:1467] Processing CreateTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 (DEFAULT_TABLE table=test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.023048 11015 tablet_service.cc:1467] Processing CreateTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 (DEFAULT_TABLE table=test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.023964 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777400e0ba7541e3b23731a49f46b4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.024194 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777400e0ba7541e3b23731a49f46b4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.024214 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 777400e0ba7541e3b23731a49f46b4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.036175 11110 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Bootstrap starting.
I20260621 06:58:40.038496 11111 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Bootstrap starting.
I20260621 06:58:40.040242 11110 tablet_bootstrap.cc:654] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.041553 11110 log.cc:824] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:40.042158 11111 tablet_bootstrap.cc:654] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.043735 11111 log.cc:824] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:40.044628 11112 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Bootstrap starting.
I20260621 06:58:40.046851 11111 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: No bootstrap required, opened a new log
I20260621 06:58:40.047147 11111 ts_tablet_manager.cc:1397] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260621 06:58:40.047739 11110 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: No bootstrap required, opened a new log
I20260621 06:58:40.048026 11110 ts_tablet_manager.cc:1397] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260621 06:58:40.048132 11112 tablet_bootstrap.cc:654] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.049414 11112 log.cc:824] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:40.054548 11112 tablet_bootstrap.cc:492] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: No bootstrap required, opened a new log
I20260621 06:58:40.054816 11112 ts_tablet_manager.cc:1397] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260621 06:58:40.056634 11111 raft_consensus.cc:348] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.057057 11111 raft_consensus.cc:374] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.057147 11111 raft_consensus.cc:729] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.057853 11111 consensus_queue.cc:260] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.058610 11110 raft_consensus.cc:348] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.059067 11110 raft_consensus.cc:374] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.059149 11110 raft_consensus.cc:729] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a183474e7484da3b52dbedbd879447f, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.059748 11110 consensus_queue.cc:260] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.066519 11111 ts_tablet_manager.cc:1428] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
I20260621 06:58:40.074684 11110 ts_tablet_manager.cc:1428] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Time spent starting tablet: real 0.026s user 0.022s sys 0.004s
I20260621 06:58:40.075664 11112 raft_consensus.cc:348] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.076187 11112 raft_consensus.cc:374] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.076292 11112 raft_consensus.cc:729] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.076920 11112 consensus_queue.cc:260] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.079645 11112 ts_tablet_manager.cc:1428] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Time spent starting tablet: real 0.025s user 0.014s sys 0.003s
I20260621 06:58:40.109187 11118 raft_consensus.cc:482] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:40.109468 11118 raft_consensus.cc:504] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.111075 11118 leader_election.cc:288] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935), 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545)
I20260621 06:58:40.118139 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.111526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:59473 (local address 127.2.245.194:39935)
0621 06:58:40.111784 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.111792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.112510 (+ 718us) server_negotiation.cc:407] Connection header received
0621 06:58:40.112804 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.112810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.112947 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.113048 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.113731 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.114434 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.116384 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.117168 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.117346 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.117424 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.117688 (+ 264us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.117901 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 06:58:40.118149 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.111582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50083 (local address 127.2.245.193:40545)
0621 06:58:40.111785 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.111794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.112622 (+ 828us) server_negotiation.cc:407] Connection header received
0621 06:58:40.112948 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.112954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.113039 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.113124 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.113888 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.114560 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.116372 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.117131 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.117317 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.117393 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.117765 (+ 372us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.117942 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 06:58:40.118237 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.111573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:40545 (local address 127.2.245.195:50083)
0621 06:58:40.112519 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.112537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.112796 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.113237 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.113241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.113265 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.113742 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.113752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.114694 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.114703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.116222 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.116235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.117347 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.117354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.117511 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.117534 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.117632 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.117772 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":163,"thread_start_us":423,"threads_started":1}
I20260621 06:58:40.118240 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.111560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:39935 (local address 127.2.245.195:59473)
0621 06:58:40.112377 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.112397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.112606 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.113063 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.113067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.113090 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.113558 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.113576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.114557 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.114564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.116237 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.116249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.117390 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.117396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.117533 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.117556 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.117671 (+ 115us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.117795 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":63,"thread_start_us":406,"threads_started":1}
I20260621 06:58:40.119342 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777400e0ba7541e3b23731a49f46b4d5" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" is_pre_election: true
I20260621 06:58:40.119342 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777400e0ba7541e3b23731a49f46b4d5" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f" is_pre_election: true
I20260621 06:58:40.119956 10898 raft_consensus.cc:2454] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:40.119938 10760 raft_consensus.cc:2454] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:40.120573 10966 leader_election.cc:302] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:40.120975 11118 raft_consensus.cc:2781] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:40.121115 11118 raft_consensus.cc:482] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:40.121222 11118 raft_consensus.cc:3037] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.123487 11118 raft_consensus.cc:504] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.124106 11118 leader_election.cc:288] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Requested vote from peers 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935), 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545)
I20260621 06:58:40.124487 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777400e0ba7541e3b23731a49f46b4d5" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
I20260621 06:58:40.124609 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "777400e0ba7541e3b23731a49f46b4d5" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f"
I20260621 06:58:40.124681 10898 raft_consensus.cc:3037] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.124826 10760 raft_consensus.cc:3037] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.126839 10898 raft_consensus.cc:2454] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:40.127085 10760 raft_consensus.cc:2454] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:40.127266 10966 leader_election.cc:302] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:40.127550 11118 raft_consensus.cc:2781] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:40.128029 11118 raft_consensus.cc:686] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Becoming Leader. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Running, Role: LEADER
I20260621 06:58:40.128479 11118 consensus_queue.cc:237] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } }
I20260621 06:58:40.133191 10600 catalog_manager.cc:5495] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195). New cstate: current_term: 1 leader_uuid: "97bcee678e0b4e0eb64532417dd4319e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } health_report { overall_health: UNKNOWN } } }
I20260621 06:58:40.149813 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.144288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.2.245.195:46051)
0621 06:58:40.144456 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.144460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.144476 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:40.144677 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.144680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.144768 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.144883 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:40.145248 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.145880 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.147719 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.148107 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.148194 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.148648 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.149195 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.149318 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.149431 (+ 113us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.149495 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:58:40.157788 10760 raft_consensus.cc:1261] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.157804 10898 raft_consensus.cc:1261] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.158772 11121 consensus_queue.cc:1035] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.159145 11118 consensus_queue.cc:1035] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.170739 11124 mvcc.cc:205] Tried to move back new op lower bound from 7299174892156891136 to 7299174892049551360. Current Snapshot: MvccSnapshot[applied={T|T < 7299174892156891136}]
I20260621 06:58:40.183707 10600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:40.183956 10600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:40.187633 10600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Sending DeleteTablet for 3 replicas of tablet 777400e0ba7541e3b23731a49f46b4d5
I20260621 06:58:40.188678 11015 tablet_service.cc:1514] Processing DeleteTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:32800
I20260621 06:58:40.188966 10737 tablet_service.cc:1514] Processing DeleteTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260621 06:58:40.189172 10876 tablet_service.cc:1514] Processing DeleteTablet for tablet 777400e0ba7541e3b23731a49f46b4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47268
I20260621 06:58:40.191461 10600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:40.192878 10600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:40.202359 11136 tablet_replica.cc:330] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: stopping tablet replica
I20260621 06:58:40.202960 10736 tablet_service.cc:1467] Processing CreateTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 (DEFAULT_TABLE table=test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.203245 11136 raft_consensus.cc:2229] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:40.203572 10736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e03381aef54d1b91aea530d8d0ea49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.203547 11014 tablet_service.cc:1467] Processing CreateTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 (DEFAULT_TABLE table=test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.203959 11136 raft_consensus.cc:2258] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.204216 11014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e03381aef54d1b91aea530d8d0ea49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.206146 11136 ts_tablet_manager.cc:1905] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.207727 10876 tablet_service.cc:1467] Processing CreateTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 (DEFAULT_TABLE table=test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.208202 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e03381aef54d1b91aea530d8d0ea49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.209977 11110 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Bootstrap starting.
I20260621 06:58:40.211651 11110 tablet_bootstrap.cc:654] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.213871 11112 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Bootstrap starting.
I20260621 06:58:40.215582 11112 tablet_bootstrap.cc:654] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.229159 11111 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Bootstrap starting.
I20260621 06:58:40.231616 11111 tablet_bootstrap.cc:654] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.234562 11112 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: No bootstrap required, opened a new log
I20260621 06:58:40.234750 11112 ts_tablet_manager.cc:1397] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.005s
I20260621 06:58:40.235417 11135 tablet_replica.cc:330] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: stopping tablet replica
I20260621 06:58:40.236006 11134 tablet_replica.cc:330] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: stopping tablet replica
I20260621 06:58:40.236617 11112 raft_consensus.cc:348] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.236843 11112 raft_consensus.cc:374] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.236938 11112 raft_consensus.cc:729] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.237061 11134 raft_consensus.cc:2229] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:40.237272 11112 consensus_queue.cc:260] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.237748 11134 raft_consensus.cc:2258] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.237860 11112 ts_tablet_manager.cc:1428] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260621 06:58:40.238570 11135 raft_consensus.cc:2229] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:40.239112 11135 raft_consensus.cc:2258] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.241047 11110 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: No bootstrap required, opened a new log
I20260621 06:58:40.241201 11110 ts_tablet_manager.cc:1397] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.002s
I20260621 06:58:40.242751 11110 raft_consensus.cc:348] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.243011 11111 tablet_bootstrap.cc:492] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: No bootstrap required, opened a new log
I20260621 06:58:40.243014 11110 raft_consensus.cc:374] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.243108 11110 raft_consensus.cc:729] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a183474e7484da3b52dbedbd879447f, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.243168 11111 ts_tablet_manager.cc:1397] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260621 06:58:40.243413 11110 consensus_queue.cc:260] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.243884 11111 raft_consensus.cc:348] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.244110 11111 raft_consensus.cc:374] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.244134 11110 ts_tablet_manager.cc:1428] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260621 06:58:40.244199 11111 raft_consensus.cc:729] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.244499 11111 consensus_queue.cc:260] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.245054 11111 ts_tablet_manager.cc:1428] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260621 06:58:40.248037 11135 ts_tablet_manager.cc:1905] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.248935 11134 ts_tablet_manager.cc:1905] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.253093 11135 ts_tablet_manager.cc:1918] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:40.253264 11135 log.cc:1196] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/777400e0ba7541e3b23731a49f46b4d5
I20260621 06:58:40.253808 11135 ts_tablet_manager.cc:1939] T 777400e0ba7541e3b23731a49f46b4d5 P 3a183474e7484da3b52dbedbd879447f: Deleting consensus metadata
I20260621 06:58:40.253770 11136 ts_tablet_manager.cc:1918] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:40.254035 11136 log.cc:1196] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/777400e0ba7541e3b23731a49f46b4d5
I20260621 06:58:40.254657 11136 ts_tablet_manager.cc:1939] T 777400e0ba7541e3b23731a49f46b4d5 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting consensus metadata
I20260621 06:58:40.255723 10586 catalog_manager.cc:4841] TS 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545): tablet 777400e0ba7541e3b23731a49f46b4d5 (table test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]) successfully deleted
I20260621 06:58:40.257472 10584 catalog_manager.cc:4841] TS 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935): tablet 777400e0ba7541e3b23731a49f46b4d5 (table test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]) successfully deleted
I20260621 06:58:40.258450 11134 ts_tablet_manager.cc:1918] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:40.258621 11134 log.cc:1196] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/777400e0ba7541e3b23731a49f46b4d5
I20260621 06:58:40.259210 11134 ts_tablet_manager.cc:1939] T 777400e0ba7541e3b23731a49f46b4d5 P 97bcee678e0b4e0eb64532417dd4319e: Deleting consensus metadata
I20260621 06:58:40.260792 10587 catalog_manager.cc:4841] TS 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051): tablet 777400e0ba7541e3b23731a49f46b4d5 (table test-table [id=9c55dde56a00488ba2f8bda9ebc29aa3]) successfully deleted
W20260621 06:58:40.291666 10808 tablet.cc:2369] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:40.301416 10947 tablet.cc:2369] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 06:58:40.322453 11086 tablet.cc:2369] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:58:40.402027 11118 raft_consensus.cc:482] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:40.402231 11118 raft_consensus.cc:504] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.402979 11118 leader_election.cc:288] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935)
I20260621 06:58:40.403641 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e03381aef54d1b91aea530d8d0ea49" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f" is_pre_election: true
I20260621 06:58:40.403631 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e03381aef54d1b91aea530d8d0ea49" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" is_pre_election: true
I20260621 06:58:40.403888 10898 raft_consensus.cc:2454] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:40.403898 10760 raft_consensus.cc:2454] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:40.404340 10968 leader_election.cc:302] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a183474e7484da3b52dbedbd879447f, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:40.404654 11118 raft_consensus.cc:2781] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:40.404780 11118 raft_consensus.cc:482] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:40.404879 11118 raft_consensus.cc:3037] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.407984 11118 raft_consensus.cc:504] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.408608 11118 leader_election.cc:288] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Requested vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935)
I20260621 06:58:40.409142 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e03381aef54d1b91aea530d8d0ea49" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
I20260621 06:58:40.409129 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e03381aef54d1b91aea530d8d0ea49" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f"
I20260621 06:58:40.409351 10760 raft_consensus.cc:3037] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.409363 10898 raft_consensus.cc:3037] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.411578 10898 raft_consensus.cc:2454] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:40.412055 10966 leader_election.cc:302] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:40.412307 10760 raft_consensus.cc:2454] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:40.412354 11118 raft_consensus.cc:2781] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:40.412551 11118 raft_consensus.cc:686] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Becoming Leader. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Running, Role: LEADER
I20260621 06:58:40.412835 11118 consensus_queue.cc:237] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.415858 10598 catalog_manager.cc:5495] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195). New cstate: current_term: 1 leader_uuid: "97bcee678e0b4e0eb64532417dd4319e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } health_report { overall_health: HEALTHY } } }
Jun 21 06:58:40 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025117, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.254@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:40.520617 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.505442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44133 (local address 127.0.0.1:43598)
0621 06:58:40.506369 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.506389 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.506555 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.506862 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.506865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.507635 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:40.507912 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.507921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.509046 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.509053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.510007 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.510017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.510700 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.510705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.510900 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.511686 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:40.511711 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:40.514371 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:40.517567 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:40.517611 (+ 44us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:40.517618 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:40.518254 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:40.518764 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:40.518772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:40.518779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:40.519015 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:40.519573 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:40.519581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:40.519778 (+ 197us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.519951 (+ 173us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.520251 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":594,"threads_started":1}
I20260621 06:58:40.520979 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.505444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.2.245.254:44133)
0621 06:58:40.506165 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.506176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.506476 (+ 300us) server_negotiation.cc:407] Connection header received
0621 06:58:40.506665 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.506670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.506738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.506842 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:40.508080 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.508894 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.510138 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.510528 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.510659 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.514584 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:40.514589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:40.514593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:40.514620 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:40.517434 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:40.518467 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:40.518476 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:40.518481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:40.518638 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:40.519116 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:40.519122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:40.519126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:40.519428 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:40.519597 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.520133 (+ 536us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.520499 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":549,"thread_start_us":348,"threads_started":1}
I20260621 06:58:40.526185 10598 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:43598:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 06:58:40.527415 10598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:40.534300 10737 tablet_service.cc:1467] Processing CreateTablet for tablet 25bd67c31aaf4f5198343a9eef41c850 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=faec914cba994b3f8153eb13f2f5f88e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:58:40.534634 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 25bd67c31aaf4f5198343a9eef41c850 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=faec914cba994b3f8153eb13f2f5f88e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:58:40.534881 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25bd67c31aaf4f5198343a9eef41c850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.535183 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25bd67c31aaf4f5198343a9eef41c850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.538745 11015 tablet_service.cc:1467] Processing CreateTablet for tablet 25bd67c31aaf4f5198343a9eef41c850 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=faec914cba994b3f8153eb13f2f5f88e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 06:58:40.539491 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25bd67c31aaf4f5198343a9eef41c850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.542286 11110 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: Bootstrap starting.
I20260621 06:58:40.544049 11110 tablet_bootstrap.cc:654] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.545543 11111 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Bootstrap starting.
I20260621 06:58:40.546288 11112 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: Bootstrap starting.
I20260621 06:58:40.547261 11111 tablet_bootstrap.cc:654] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.547895 11112 tablet_bootstrap.cc:654] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.549042 11110 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: No bootstrap required, opened a new log
I20260621 06:58:40.549233 11110 ts_tablet_manager.cc:1397] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260621 06:58:40.549916 11112 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: No bootstrap required, opened a new log
I20260621 06:58:40.550077 11112 ts_tablet_manager.cc:1397] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260621 06:58:40.550127 11110 raft_consensus.cc:348] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.550374 11110 raft_consensus.cc:374] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.550449 11110 raft_consensus.cc:729] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a183474e7484da3b52dbedbd879447f, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.550664 11110 consensus_queue.cc:260] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.550716 11112 raft_consensus.cc:348] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.550985 11112 raft_consensus.cc:374] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.551102 11112 raft_consensus.cc:729] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.551141 11117 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.551357 11112 consensus_queue.cc:260] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.551514 11117 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.551972 11112 ts_tablet_manager.cc:1428] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260621 06:58:40.552502 11110 ts_tablet_manager.cc:1428] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260621 06:58:40.551764 11118 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.554914 11118 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.560371 11111 tablet_bootstrap.cc:492] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: No bootstrap required, opened a new log
I20260621 06:58:40.560549 11111 ts_tablet_manager.cc:1397] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260621 06:58:40.561319 11111 raft_consensus.cc:348] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.561591 11111 raft_consensus.cc:374] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.561669 11111 raft_consensus.cc:729] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.561900 11111 consensus_queue.cc:260] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.562544 11111 ts_tablet_manager.cc:1428] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260621 06:58:40.562417 11116 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.562851 11116 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.623556 11116 raft_consensus.cc:482] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:40.623787 11116 raft_consensus.cc:504] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.625258 11116 leader_election.cc:288] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051)
I20260621 06:58:40.644511 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.625948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:55277 (local address 127.2.245.195:46051)
0621 06:58:40.626150 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.626158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.637387 (+ 11229us) server_negotiation.cc:407] Connection header received
0621 06:58:40.637596 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.637601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.637693 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.637772 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.638525 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.639346 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.642237 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.643496 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.643649 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.643720 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.644048 (+ 328us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.644183 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 06:58:40.645155 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.625555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:40545 (local address 127.2.245.194:44263)
0621 06:58:40.636864 (+ 11309us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.636888 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.637082 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.637530 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.637538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.637574 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.637936 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.637950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.639564 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.639569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.642054 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.642069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.644344 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.644350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.644517 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.644559 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.644647 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.644770 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11203,"mutex_wait_us":126,"thread_start_us":11123,"threads_started":1}
I20260621 06:58:40.645515 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.625654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:44263 (local address 127.2.245.193:40545)
0621 06:58:40.636438 (+ 10784us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.636445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.637224 (+ 779us) server_negotiation.cc:407] Connection header received
0621 06:58:40.637311 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.637315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.637392 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.637496 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:58:40.638193 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.639371 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.643058 (+ 3687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.643985 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.644132 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.644206 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.644868 (+ 662us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.645009 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10635,"thread_start_us":10777,"threads_started":1}
I20260621 06:58:40.645637 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.625604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46051 (local address 127.2.245.194:55277)
0621 06:58:40.637281 (+ 11677us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.637295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.637420 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.637977 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.637982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.638012 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:58:40.638345 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.638353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.639469 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.639476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.642052 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.642069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.643658 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.643674 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.643847 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.643883 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.644860 (+ 977us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.644984 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11574,"mutex_wait_us":99,"thread_start_us":11385,"threads_started":1}
I20260621 06:58:40.646492 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25bd67c31aaf4f5198343a9eef41c850" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f" is_pre_election: true
I20260621 06:58:40.646543 11040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25bd67c31aaf4f5198343a9eef41c850" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bcee678e0b4e0eb64532417dd4319e" is_pre_election: true
I20260621 06:58:40.646786 10760 raft_consensus.cc:2454] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 0.
I20260621 06:58:40.646875 11040 raft_consensus.cc:2454] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 0.
I20260621 06:58:40.647377 10829 leader_election.cc:302] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 3a183474e7484da3b52dbedbd879447f; no voters:
I20260621 06:58:40.647800 11116 raft_consensus.cc:2781] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:40.647923 11116 raft_consensus.cc:482] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:40.648025 11116 raft_consensus.cc:3037] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.650290 11116 raft_consensus.cc:504] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.650851 11116 leader_election.cc:288] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 election: Requested vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051)
I20260621 06:58:40.651279 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25bd67c31aaf4f5198343a9eef41c850" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f"
I20260621 06:58:40.651512 10760 raft_consensus.cc:3037] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.651448 11040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25bd67c31aaf4f5198343a9eef41c850" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bcee678e0b4e0eb64532417dd4319e"
I20260621 06:58:40.651682 11040 raft_consensus.cc:3037] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:40.654664 10760 raft_consensus.cc:2454] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 1.
I20260621 06:58:40.654724 11040 raft_consensus.cc:2454] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 1.
I20260621 06:58:40.655133 10829 leader_election.cc:302] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 3a183474e7484da3b52dbedbd879447f; no voters:
I20260621 06:58:40.655542 11116 raft_consensus.cc:2781] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:40.656073 11116 raft_consensus.cc:686] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 LEADER]: Becoming Leader. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Running, Role: LEADER
I20260621 06:58:40.656472 11116 consensus_queue.cc:237] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.658345 11154 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: TxnStatusTablet state changed. Reason: New leader 031265424c7b41ea8c1cb4ba3fe2ac8b. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.658726 11154 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:58:40.659567 11156 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 06:58:40.659781 11156 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 06:58:40.660159 10598 catalog_manager.cc:5495] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194). New cstate: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } health_report { overall_health: UNKNOWN } } }
I20260621 06:58:40.683723 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.678065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:39935 (local address 127.0.0.1:47284)
0621 06:58:40.678278 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.678300 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.678439 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.678983 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.678988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.678999 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:58:40.679260 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.679270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.680331 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.680338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.681335 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.681347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.682104 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.682109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.682260 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.682311 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.683163 (+ 852us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.683182 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.683292 (+ 110us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.683444 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260621 06:58:40.683769 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.678092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.2.245.194:39935)
0621 06:58:40.678598 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.678609 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.678624 (+ 15us) server_negotiation.cc:407] Connection header received
0621 06:58:40.678729 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.678733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.678798 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.678946 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:40.679386 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.680205 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.681475 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.681936 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.682088 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.682496 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.683059 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.683190 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.683336 (+ 146us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.683419 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":229,"threads_started":1}
I20260621 06:58:40.688511 10760 raft_consensus.cc:1261] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Refusing update from remote peer 031265424c7b41ea8c1cb4ba3fe2ac8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.689210 11154 consensus_queue.cc:1035] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.689412 11040 raft_consensus.cc:1261] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Refusing update from remote peer 031265424c7b41ea8c1cb4ba3fe2ac8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.690112 11154 consensus_queue.cc:1035] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.700512 11118 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: TxnStatusTablet state changed. Reason: New leader 031265424c7b41ea8c1cb4ba3fe2ac8b. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.700793 11118 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.702085 11117 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: TxnStatusTablet state changed. Reason: New leader 031265424c7b41ea8c1cb4ba3fe2ac8b. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.702417 11117 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.707125 11160 mvcc.cc:205] Tried to move back new op lower bound from 7299174894332432384 to 7299174894212177920. Current Snapshot: MvccSnapshot[applied={T|T < 7299174894332432384}]
I20260621 06:58:40.712323 11154 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.712661 11154 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:58:40.712852 11116 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.713156 11116 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 031265424c7b41ea8c1cb4ba3fe2ac8b: This TxnStatusTablet replica's current role is: LEADER
I20260621 06:58:40.719908 11117 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.720171 11117 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 3a183474e7484da3b52dbedbd879447f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.722232 11118 tablet_replica.cc:439] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } } }
I20260621 06:58:40.722537 11118 tablet_replica.cc:442] T 25bd67c31aaf4f5198343a9eef41c850 P 97bcee678e0b4e0eb64532417dd4319e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 06:58:40.734530 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.728449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:39935 (local address 127.2.245.195:57989)
0621 06:58:40.729112 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.729128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.729269 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.729641 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.729644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.729654 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:58:40.729914 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.729922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.731505 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.731509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.732513 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.732525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.733492 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.733496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.733607 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.733658 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.734050 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.734057 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.734133 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.734254 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":328,"threads_started":1}
I20260621 06:58:40.734524 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.728456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:57989 (local address 127.2.245.194:39935)
0621 06:58:40.728629 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.728635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.729234 (+ 599us) server_negotiation.cc:407] Connection header received
0621 06:58:40.729407 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.729413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.729497 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.729608 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:40.730213 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.731378 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.732645 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.733320 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.733451 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.733825 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.733952 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.734058 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.734195 (+ 137us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.734272 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:40.743120 10898 raft_consensus.cc:1261] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.743196 10760 raft_consensus.cc:1261] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:40.743642 11118 consensus_queue.cc:1035] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.743933 11129 consensus_queue.cc:1035] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:40.786196 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.771328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:35477 (local address 127.2.245.195:46051)
0621 06:58:40.771486 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.771492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.779632 (+ 8140us) server_negotiation.cc:407] Connection header received
0621 06:58:40.779855 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.779859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.779947 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.780046 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:40.780668 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.781775 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.784062 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.784684 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.784873 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.785389 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.785518 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.785605 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.785823 (+ 218us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.785908 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 06:58:40.787029 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.770973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46051 (local address 127.2.245.194:35477)
0621 06:58:40.779520 (+ 8547us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:40.779540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:40.779703 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:40.780089 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:40.780094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:40.780106 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:58:40.780476 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.780488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.781957 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.781963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.783883 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:40.783900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.784913 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.784920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:40.785128 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.785202 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:58:40.785674 (+ 472us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:58:40.785696 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 06:58:40.786401 (+ 705us) client_negotiation.cc:240] Negotiation successful
0621 06:58:40.786550 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8340,"thread_start_us":8135,"threads_started":1}
I20260621 06:58:40.830677 10598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:40.830942 10598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:40.833981 10598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Sending DeleteTablet for 3 replicas of tablet b0e03381aef54d1b91aea530d8d0ea49
I20260621 06:58:40.834779 10737 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260621 06:58:40.834985 11015 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:32800
I20260621 06:58:40.835583 11179 tablet_replica.cc:330] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: stopping tablet replica
I20260621 06:58:40.835846 11179 raft_consensus.cc:2229] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:40.835953 10876 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e03381aef54d1b91aea530d8d0ea49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:40 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47268
I20260621 06:58:40.836383 11179 raft_consensus.cc:2258] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.837678 11179 ts_tablet_manager.cc:1905] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.846458 11179 ts_tablet_manager.cc:1918] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:40.846714 11179 log.cc:1196] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/b0e03381aef54d1b91aea530d8d0ea49
I20260621 06:58:40.847316 11179 ts_tablet_manager.cc:1939] T b0e03381aef54d1b91aea530d8d0ea49 P 97bcee678e0b4e0eb64532417dd4319e: Deleting consensus metadata
I20260621 06:58:40.848640 10587 catalog_manager.cc:4841] TS 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051): tablet b0e03381aef54d1b91aea530d8d0ea49 (table test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]) successfully deleted
I20260621 06:58:40.851532 11180 tablet_replica.cc:330] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: stopping tablet replica
I20260621 06:58:40.851892 11180 raft_consensus.cc:2229] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:40.852325 11180 raft_consensus.cc:2258] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.852974 11178 tablet_replica.cc:330] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: stopping tablet replica
I20260621 06:58:40.853184 11178 raft_consensus.cc:2229] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:40.853430 11178 raft_consensus.cc:2258] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:40.854212 11180 ts_tablet_manager.cc:1905] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.859514 11180 ts_tablet_manager.cc:1918] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:40.859656 11180 log.cc:1196] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/b0e03381aef54d1b91aea530d8d0ea49
I20260621 06:58:40.859927 11178 ts_tablet_manager.cc:1905] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:40.860184 11180 ts_tablet_manager.cc:1939] T b0e03381aef54d1b91aea530d8d0ea49 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting consensus metadata
I20260621 06:58:40.861339 10584 catalog_manager.cc:4841] TS 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935): tablet b0e03381aef54d1b91aea530d8d0ea49 (table test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]) successfully deleted
I20260621 06:58:40.867612 11178 ts_tablet_manager.cc:1918] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:40.867825 11178 log.cc:1196] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/b0e03381aef54d1b91aea530d8d0ea49
I20260621 06:58:40.868314 11178 ts_tablet_manager.cc:1939] T b0e03381aef54d1b91aea530d8d0ea49 P 3a183474e7484da3b52dbedbd879447f: Deleting consensus metadata
I20260621 06:58:40.869418 10586 catalog_manager.cc:4841] TS 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545): tablet b0e03381aef54d1b91aea530d8d0ea49 (table test-table [id=b8d7e46ad4d74453b7a64317c811d6d1]) successfully deleted
Jun 21 06:58:40 dist-test-slave-9zdv krb5kdc[10555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 06:58:40 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:40 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:40.911629 11146 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:40.900489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.2.245.254:44133)
0621 06:58:40.900701 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.900709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.901078 (+ 369us) server_negotiation.cc:407] Connection header received
0621 06:58:40.901281 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.901290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.901362 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.901466 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:40.902750 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.903542 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.904919 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.905422 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.905561 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.911368 (+ 5807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43610: BlockingRecv error: recv got EOF from 127.0.0.1:43610 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 21 06:58:40 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM
I20260621 06:58:40.930457 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:40.914507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43626 (local address 127.2.245.254:44133)
0621 06:58:40.914673 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:40.914677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:40.914976 (+ 299us) server_negotiation.cc:407] Connection header received
0621 06:58:40.915138 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:40.915142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:40.915200 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:40.915284 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:40.916748 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.917448 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.919304 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:40.919747 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:40.919841 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:40.924393 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:40.924401 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:40.924406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:40.924449 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:40.927730 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:40.928359 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:40.928365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:40.928369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:40.928532 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:40.928978 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:40.928984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:40.928988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:40.929343 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:40.929595 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:40.929822 (+ 227us) server_negotiation.cc:299] Negotiation successful
0621 06:58:40.930055 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 06:58:40.934141 10598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:40.934787 10598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:40.941784 10737 tablet_service.cc:1467] Processing CreateTablet for tablet 45c885f826984766832562a863212cd8 (DEFAULT_TABLE table=test-table [id=7bc84e3d72d341b5bfd68999caf92930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.941790 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 45c885f826984766832562a863212cd8 (DEFAULT_TABLE table=test-table [id=7bc84e3d72d341b5bfd68999caf92930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.941936 11015 tablet_service.cc:1467] Processing CreateTablet for tablet 45c885f826984766832562a863212cd8 (DEFAULT_TABLE table=test-table [id=7bc84e3d72d341b5bfd68999caf92930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:40.942214 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45c885f826984766832562a863212cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.942237 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45c885f826984766832562a863212cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.942343 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45c885f826984766832562a863212cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:40.948807 11110 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Bootstrap starting.
I20260621 06:58:40.948916 11111 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Bootstrap starting.
I20260621 06:58:40.950521 11111 tablet_bootstrap.cc:654] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.950517 11110 tablet_bootstrap.cc:654] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.951694 11112 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Bootstrap starting.
I20260621 06:58:40.953799 11112 tablet_bootstrap.cc:654] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:40.956288 11111 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: No bootstrap required, opened a new log
I20260621 06:58:40.957480 11111 ts_tablet_manager.cc:1397] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260621 06:58:40.958326 11111 raft_consensus.cc:348] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.958704 11111 raft_consensus.cc:374] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.958843 11111 raft_consensus.cc:729] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.959153 11111 consensus_queue.cc:260] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.959781 11111 ts_tablet_manager.cc:1428] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260621 06:58:40.962616 11110 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: No bootstrap required, opened a new log
I20260621 06:58:40.962795 11110 ts_tablet_manager.cc:1397] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260621 06:58:40.963685 11110 raft_consensus.cc:348] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.963958 11110 raft_consensus.cc:374] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.964056 11110 raft_consensus.cc:729] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a183474e7484da3b52dbedbd879447f, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.964303 11110 consensus_queue.cc:260] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.965045 11112 tablet_bootstrap.cc:492] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: No bootstrap required, opened a new log
I20260621 06:58:40.965214 11112 ts_tablet_manager.cc:1397] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.007s
I20260621 06:58:40.965597 11110 ts_tablet_manager.cc:1428] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260621 06:58:40.967142 11112 raft_consensus.cc:348] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.967414 11112 raft_consensus.cc:374] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:40.967510 11112 raft_consensus.cc:729] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Initialized, Role: FOLLOWER
I20260621 06:58:40.967806 11112 consensus_queue.cc:260] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:40.969218 11112 ts_tablet_manager.cc:1428] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260621 06:58:41.021056 11129 raft_consensus.cc:482] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:41.021302 11129 raft_consensus.cc:504] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.021981 11129 leader_election.cc:288] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935)
I20260621 06:58:41.022563 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45c885f826984766832562a863212cd8" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f" is_pre_election: true
I20260621 06:58:41.022666 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45c885f826984766832562a863212cd8" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" is_pre_election: true
I20260621 06:58:41.022827 10760 raft_consensus.cc:2454] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:41.022980 10898 raft_consensus.cc:2454] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 0.
I20260621 06:58:41.023346 10968 leader_election.cc:302] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a183474e7484da3b52dbedbd879447f, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:41.023639 11129 raft_consensus.cc:2781] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:41.023749 11129 raft_consensus.cc:482] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:41.023880 11129 raft_consensus.cc:3037] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.025952 11129 raft_consensus.cc:504] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.026563 11129 leader_election.cc:288] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Requested vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935)
I20260621 06:58:41.027032 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45c885f826984766832562a863212cd8" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f"
I20260621 06:58:41.027086 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45c885f826984766832562a863212cd8" candidate_uuid: "97bcee678e0b4e0eb64532417dd4319e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b"
I20260621 06:58:41.027261 10760 raft_consensus.cc:3037] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.027297 10898 raft_consensus.cc:3037] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.030293 10760 raft_consensus.cc:2454] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:41.030298 10898 raft_consensus.cc:2454] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bcee678e0b4e0eb64532417dd4319e in term 1.
I20260621 06:58:41.030736 10968 leader_election.cc:302] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a183474e7484da3b52dbedbd879447f, 97bcee678e0b4e0eb64532417dd4319e; no voters:
I20260621 06:58:41.031067 11129 raft_consensus.cc:2781] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:41.031261 11129 raft_consensus.cc:686] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Becoming Leader. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Running, Role: LEADER
I20260621 06:58:41.031522 11129 consensus_queue.cc:237] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.034243 10600 catalog_manager.cc:5495] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195). New cstate: current_term: 1 leader_uuid: "97bcee678e0b4e0eb64532417dd4319e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } health_report { overall_health: HEALTHY } } }
I20260621 06:58:41.061152 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.055366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.2.245.195:46051)
0621 06:58:41.055573 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.055579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.055765 (+ 186us) server_negotiation.cc:407] Connection header received
0621 06:58:41.055956 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.055962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.056050 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.056163 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:41.056535 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.057615 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.058993 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.059706 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.059846 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.060354 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:41.060491 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:41.060591 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.060760 (+ 169us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.060844 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:41.064015 10898 raft_consensus.cc:1261] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:41.064101 10760 raft_consensus.cc:1261] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Refusing update from remote peer 97bcee678e0b4e0eb64532417dd4319e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:41.064657 11129 consensus_queue.cc:1035] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:41.065017 11118 consensus_queue.cc:1035] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:41.072220 11161 mvcc.cc:205] Tried to move back new op lower bound from 7299174895872192512 to 7299174895747444736. Current Snapshot: MvccSnapshot[applied={T|T < 7299174895872192512}]
I20260621 06:58:41.078598 10600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:41.078986 10600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:41.083257 10600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Sending DeleteTablet for 3 replicas of tablet 45c885f826984766832562a863212cd8
I20260621 06:58:41.084272 10737 tablet_service.cc:1514] Processing DeleteTablet for tablet 45c885f826984766832562a863212cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260621 06:58:41.084540 11178 tablet_replica.cc:330] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: stopping tablet replica
I20260621 06:58:41.084575 11015 tablet_service.cc:1514] Processing DeleteTablet for tablet 45c885f826984766832562a863212cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:32800
I20260621 06:58:41.084728 11178 raft_consensus.cc:2229] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:41.084954 11178 raft_consensus.cc:2258] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.085270 11014 tablet_service.cc:1514] Processing DeleteTablet for tablet 45c885f826984766832562a863212cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:32800
I20260621 06:58:41.085500 11179 tablet_replica.cc:330] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: stopping tablet replica
I20260621 06:58:41.085631 11179 raft_consensus.cc:2229] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:41.085868 11179 raft_consensus.cc:2258] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.086002 10875 tablet_service.cc:1514] Processing DeleteTablet for tablet 45c885f826984766832562a863212cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47268
I20260621 06:58:41.086371 10876 tablet_service.cc:1514] Processing DeleteTablet for tablet 45c885f826984766832562a863212cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47268
I20260621 06:58:41.086560 11180 tablet_replica.cc:330] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: stopping tablet replica
I20260621 06:58:41.086697 11180 raft_consensus.cc:2229] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:41.086895 11180 raft_consensus.cc:2258] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.087145 10598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:58:41.087834 10598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:58:41.088898 11178 ts_tablet_manager.cc:1905] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.095597 11179 ts_tablet_manager.cc:1905] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.096171 11014 tablet_service.cc:1467] Processing CreateTablet for tablet 7e907fa201154db48574f711b4b6c807 (DEFAULT_TABLE table=test-table [id=f81e15a097f846f8aaefa3e0d218e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:41.096648 11014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e907fa201154db48574f711b4b6c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:41.099370 10737 tablet_service.cc:1467] Processing CreateTablet for tablet 7e907fa201154db48574f711b4b6c807 (DEFAULT_TABLE table=test-table [id=f81e15a097f846f8aaefa3e0d218e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:41.099824 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e907fa201154db48574f711b4b6c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:41.102170 11179 ts_tablet_manager.cc:1918] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:41.102316 11179 log.cc:1196] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/45c885f826984766832562a863212cd8
I20260621 06:58:41.102706 11179 ts_tablet_manager.cc:1939] T 45c885f826984766832562a863212cd8 P 97bcee678e0b4e0eb64532417dd4319e: Deleting consensus metadata
I20260621 06:58:41.103891 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 7e907fa201154db48574f711b4b6c807 (DEFAULT_TABLE table=test-table [id=f81e15a097f846f8aaefa3e0d218e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:58:41.104362 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e907fa201154db48574f711b4b6c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:41.104527 11112 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Bootstrap starting.
I20260621 06:58:41.106189 11112 tablet_bootstrap.cc:654] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:41.109447 11110 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Bootstrap starting.
I20260621 06:58:41.111088 11110 tablet_bootstrap.cc:654] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:41.094882 11178 ts_tablet_manager.cc:1918] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:41.111625 11178 log.cc:1196] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/45c885f826984766832562a863212cd8
I20260621 06:58:41.112071 11178 ts_tablet_manager.cc:1939] T 45c885f826984766832562a863212cd8 P 3a183474e7484da3b52dbedbd879447f: Deleting consensus metadata
I20260621 06:58:41.113111 11111 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Bootstrap starting.
I20260621 06:58:41.113130 10586 catalog_manager.cc:4841] TS 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545): tablet 45c885f826984766832562a863212cd8 (table test-table [id=7bc84e3d72d341b5bfd68999caf92930]) successfully deleted
I20260621 06:58:41.115197 11111 tablet_bootstrap.cc:654] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:41.117472 10587 catalog_manager.cc:4841] TS 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051): tablet 45c885f826984766832562a863212cd8 (table test-table [id=7bc84e3d72d341b5bfd68999caf92930]) successfully deleted
I20260621 06:58:41.118412 11111 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: No bootstrap required, opened a new log
W20260621 06:58:41.118465 10587 catalog_manager.cc:4808] TS 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051): delete failed for tablet 45c885f826984766832562a863212cd8 because the tablet was not found. No further retry: Not found: Tablet not found: 45c885f826984766832562a863212cd8
I20260621 06:58:41.118641 11111 ts_tablet_manager.cc:1397] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 06:58:41.119637 11180 ts_tablet_manager.cc:1905] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.119539 11111 raft_consensus.cc:348] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.119928 11111 raft_consensus.cc:374] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:41.120018 11111 raft_consensus.cc:729] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Initialized, Role: FOLLOWER
I20260621 06:58:41.120309 11111 consensus_queue.cc:260] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.120841 11111 ts_tablet_manager.cc:1428] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260621 06:58:41.126816 11180 ts_tablet_manager.cc:1918] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:58:41.127055 11180 log.cc:1196] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/45c885f826984766832562a863212cd8
I20260621 06:58:41.127452 11180 ts_tablet_manager.cc:1939] T 45c885f826984766832562a863212cd8 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting consensus metadata
I20260621 06:58:41.128422 10584 catalog_manager.cc:4841] TS 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935): tablet 45c885f826984766832562a863212cd8 (table test-table [id=7bc84e3d72d341b5bfd68999caf92930]) successfully deleted
W20260621 06:58:41.128896 10584 catalog_manager.cc:4808] TS 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935): delete failed for tablet 45c885f826984766832562a863212cd8 because the tablet was not found. No further retry: Not found: Tablet not found: 45c885f826984766832562a863212cd8
I20260621 06:58:41.130574 11110 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: No bootstrap required, opened a new log
I20260621 06:58:41.130755 11110 ts_tablet_manager.cc:1397] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.007s
I20260621 06:58:41.131630 11110 raft_consensus.cc:348] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.131906 11110 raft_consensus.cc:374] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:41.131999 11110 raft_consensus.cc:729] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a183474e7484da3b52dbedbd879447f, State: Initialized, Role: FOLLOWER
I20260621 06:58:41.132244 11110 consensus_queue.cc:260] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.132815 11110 ts_tablet_manager.cc:1428] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260621 06:58:41.133670 11112 tablet_bootstrap.cc:492] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: No bootstrap required, opened a new log
I20260621 06:58:41.133848 11112 ts_tablet_manager.cc:1397] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.006s
I20260621 06:58:41.134646 11112 raft_consensus.cc:348] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.134882 11112 raft_consensus.cc:374] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:41.134991 11112 raft_consensus.cc:729] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bcee678e0b4e0eb64532417dd4319e, State: Initialized, Role: FOLLOWER
I20260621 06:58:41.135222 11112 consensus_queue.cc:260] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.136711 11112 ts_tablet_manager.cc:1428] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260621 06:58:41.164565 11116 raft_consensus.cc:482] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:58:41.164848 11116 raft_consensus.cc:504] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.165534 11116 leader_election.cc:288] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051)
I20260621 06:58:41.166034 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e907fa201154db48574f711b4b6c807" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f" is_pre_election: true
I20260621 06:58:41.166074 11040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e907fa201154db48574f711b4b6c807" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bcee678e0b4e0eb64532417dd4319e" is_pre_election: true
I20260621 06:58:41.166307 10760 raft_consensus.cc:2454] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 0.
I20260621 06:58:41.166366 11040 raft_consensus.cc:2454] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 0.
I20260621 06:58:41.166771 10829 leader_election.cc:302] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 3a183474e7484da3b52dbedbd879447f; no voters:
I20260621 06:58:41.167107 11116 raft_consensus.cc:2781] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:58:41.167265 11116 raft_consensus.cc:482] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:58:41.167413 11116 raft_consensus.cc:3037] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.169529 11116 raft_consensus.cc:504] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.170104 11116 leader_election.cc:288] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 election: Requested vote from peers 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545), 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051)
I20260621 06:58:41.170506 10760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e907fa201154db48574f711b4b6c807" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a183474e7484da3b52dbedbd879447f"
I20260621 06:58:41.170605 11040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e907fa201154db48574f711b4b6c807" candidate_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bcee678e0b4e0eb64532417dd4319e"
I20260621 06:58:41.170750 10760 raft_consensus.cc:3037] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.170816 11040 raft_consensus.cc:3037] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:41.173796 11040 raft_consensus.cc:2454] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 1.
I20260621 06:58:41.173794 10760 raft_consensus.cc:2454] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031265424c7b41ea8c1cb4ba3fe2ac8b in term 1.
I20260621 06:58:41.174193 10829 leader_election.cc:302] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031265424c7b41ea8c1cb4ba3fe2ac8b, 3a183474e7484da3b52dbedbd879447f; no voters:
I20260621 06:58:41.174522 11116 raft_consensus.cc:2781] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:41.174768 11116 raft_consensus.cc:686] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 LEADER]: Becoming Leader. State: Replica: 031265424c7b41ea8c1cb4ba3fe2ac8b, State: Running, Role: LEADER
I20260621 06:58:41.175105 11116 consensus_queue.cc:237] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } }
I20260621 06:58:41.177872 10598 catalog_manager.cc:5495] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194). New cstate: current_term: 1 leader_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "031265424c7b41ea8c1cb4ba3fe2ac8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 39935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 } health_report { overall_health: UNKNOWN } } }
I20260621 06:58:41.215886 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.211026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.2.245.194:39935)
0621 06:58:41.211236 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.211244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.211403 (+ 159us) server_negotiation.cc:407] Connection header received
0621 06:58:41.211538 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.211541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.211601 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.211679 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:41.211991 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.212726 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.213995 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.214555 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.214677 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.215194 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:41.215289 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:41.215356 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.215458 (+ 102us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.215518 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 06:58:41.227003 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.223098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:49217 (local address 127.2.245.194:39935)
0621 06:58:41.223298 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.223303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.223321 (+ 18us) server_negotiation.cc:407] Connection header received
0621 06:58:41.223448 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.223451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.223505 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.223573 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:58:41.223871 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.224504 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.225636 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.226037 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.226135 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.226437 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:58:41.226523 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:58:41.226585 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.226687 (+ 102us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.226742 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":70}
I20260621 06:58:41.227056 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.223110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:39935 (local address 127.2.245.194:49217)
0621 06:58:41.223253 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:41.223268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:41.223348 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:41.223594 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:41.223596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:41.223604 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 06:58:41.223764 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:41.223769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.224612 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.224615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:41.225527 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:41.225537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.226162 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.226165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:41.226287 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.226320 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 06:58:41.226597 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 06:58:41.226604 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:58:41.226658 (+ 54us) client_negotiation.cc:240] Negotiation successful
0621 06:58:41.226758 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":2}
I20260621 06:58:41.234094 10760 raft_consensus.cc:1261] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Refusing update from remote peer 031265424c7b41ea8c1cb4ba3fe2ac8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:41.234267 11040 raft_consensus.cc:1261] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Refusing update from remote peer 031265424c7b41ea8c1cb4ba3fe2ac8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:58:41.234618 11116 consensus_queue.cc:1035] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a183474e7484da3b52dbedbd879447f" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:41.234882 11154 consensus_queue.cc:1035] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97bcee678e0b4e0eb64532417dd4319e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:58:41.280361 10598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:41.280560 10598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:58:41.283622 10598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a58ace09664d47e2b8fb2eb8424fa8e5: Sending DeleteTablet for 3 replicas of tablet 7e907fa201154db48574f711b4b6c807
I20260621 06:58:41.284530 10737 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e907fa201154db48574f711b4b6c807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260621 06:58:41.284705 11014 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e907fa201154db48574f711b4b6c807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:32800
I20260621 06:58:41.284900 11178 tablet_replica.cc:330] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: stopping tablet replica
I20260621 06:58:41.284987 11179 tablet_replica.cc:330] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: stopping tablet replica
I20260621 06:58:41.285089 11178 raft_consensus.cc:2229] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:41.285168 11179 raft_consensus.cc:2229] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:58:41.285320 11178 raft_consensus.cc:2258] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.285362 11179 raft_consensus.cc:2258] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.286594 11178 ts_tablet_manager.cc:1905] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.286631 11179 ts_tablet_manager.cc:1905] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.286747 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10671
I20260621 06:58:41.291572 10875 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e907fa201154db48574f711b4b6c807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:58:41 UTC) from {username='oryx', principal='oryx/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47268
I20260621 06:58:41.291939 11180 tablet_replica.cc:330] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: stopping tablet replica
I20260621 06:58:41.292164 11180 raft_consensus.cc:2229] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 LEADER]: Raft consensus shutting down.
I20260621 06:58:41.292848 11180 raft_consensus.cc:2258] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:58:41.297549 11178 ts_tablet_manager.cc:1918] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:41.297696 11178 log.cc:1196] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/7e907fa201154db48574f711b4b6c807
I20260621 06:58:41.298146 11178 ts_tablet_manager.cc:1939] T 7e907fa201154db48574f711b4b6c807 P 3a183474e7484da3b52dbedbd879447f: Deleting consensus metadata
I20260621 06:58:41.300191 11180 ts_tablet_manager.cc:1905] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:58:41.303319 10586 catalog_manager.cc:4841] TS 3a183474e7484da3b52dbedbd879447f (127.2.245.193:40545): tablet 7e907fa201154db48574f711b4b6c807 (table test-table [id=f81e15a097f846f8aaefa3e0d218e75b]) successfully deleted
I20260621 06:58:41.306998 11179 ts_tablet_manager.cc:1918] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:41.307152 11179 log.cc:1196] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/7e907fa201154db48574f711b4b6c807
I20260621 06:58:41.307642 11179 ts_tablet_manager.cc:1939] T 7e907fa201154db48574f711b4b6c807 P 97bcee678e0b4e0eb64532417dd4319e: Deleting consensus metadata
I20260621 06:58:41.308810 10587 catalog_manager.cc:4841] TS 97bcee678e0b4e0eb64532417dd4319e (127.2.245.195:46051): tablet 7e907fa201154db48574f711b4b6c807 (table test-table [id=f81e15a097f846f8aaefa3e0d218e75b]) successfully deleted
I20260621 06:58:41.310060 11180 ts_tablet_manager.cc:1918] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 06:58:41.310214 11180 log.cc:1196] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/7e907fa201154db48574f711b4b6c807
I20260621 06:58:41.310702 11180 ts_tablet_manager.cc:1939] T 7e907fa201154db48574f711b4b6c807 P 031265424c7b41ea8c1cb4ba3fe2ac8b: Deleting consensus metadata
I20260621 06:58:41.312716 10584 catalog_manager.cc:4841] TS 031265424c7b41ea8c1cb4ba3fe2ac8b (127.2.245.194:39935): tablet 7e907fa201154db48574f711b4b6c807 (table test-table [id=f81e15a097f846f8aaefa3e0d218e75b]) successfully deleted
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.193@KRBTEST.COM
I20260621 06:58:41.326851 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.288329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.2.245.193:40545)
0621 06:58:41.311730 (+ 23401us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.311736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.311755 (+ 19us) server_negotiation.cc:407] Connection header received
0621 06:58:41.311811 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.311816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.311901 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.312007 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:41.313943 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.314611 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.315972 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.316412 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.316542 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.320806 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:41.320821 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:41.320837 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:41.320869 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:41.323944 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.324612 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.324619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.324622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.324748 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.325186 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.325190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.325192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.325593 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:41.325915 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.326200 (+ 285us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.326456 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23249,"thread_start_us":23821,"threads_started":1}
I20260621 06:58:41.327510 10802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:41.425976 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10671
I20260621 06:58:41.444675 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10811
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.194@KRBTEST.COM
I20260621 06:58:41.460505 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.445648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.2.245.194:39935)
0621 06:58:41.445867 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.445872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.445888 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:41.445937 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.445941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.446015 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.446130 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:41.447420 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.448157 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.449380 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.449897 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.450045 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.454193 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:41.454207 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:41.454224 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:41.454258 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:41.457465 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.458142 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.458149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.458152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.458297 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.458659 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.458664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.458667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.459303 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:41.459626 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.459904 (+ 278us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.460119 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 06:58:41.461282 10941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:41.605934 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10811
I20260621 06:58:41.628309 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10950
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[10555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.245.195@KRBTEST.COM
I20260621 06:58:41.643498 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:41.629071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.2.245.195:46051)
0621 06:58:41.629557 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:41.629561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:41.629575 (+ 14us) server_negotiation.cc:407] Connection header received
0621 06:58:41.629625 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:41.629629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:41.629683 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:41.629803 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:41.631065 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.631766 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.633014 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:41.633411 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:41.633549 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:41.637715 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:41.637736 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:41.637758 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:41.637795 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:41.640737 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.641430 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.641438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.641440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.641574 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:41.641929 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:41.641936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:41.641939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:41.642313 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:41.642610 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:41.642906 (+ 296us) server_negotiation.cc:299] Negotiation successful
0621 06:58:41.643166 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":237,"threads_started":1}
I20260621 06:58:41.644201 11080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:41.768201 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10950
I20260621 06:58:41.789757 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 10568
I20260621 06:58:41.790863 10640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:41.905104 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 10568
2026-06-21T06:58:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6493 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[11216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[11216](info): set up 2 sockets
Jun 21 06:58:41 dist-test-slave-9zdv krb5kdc[11216](info): commencing operation
krb5kdc: starting...
W20260621 06:58:42.993064 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:43 dist-test-slave-9zdv krb5kdc[11216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:43.208778 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44021
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:43.533336 11229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:43.533632 11229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:43.533790 11229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:43.541074 11229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:43.541175 11229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:43.541276 11229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:43.541393 11229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:43.541473 11229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:43.551972 11229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:43.553272 11229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:43.554524 11229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:43.566213 11234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:43.566233 11235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:43.566727 11237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:43.567155 11229 server_base.cc:1034] running on GCE node
I20260621 06:58:43.567948 11229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:43.569832 11229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:43.571036 11229 hybrid_clock.cc:648] HybridClock initialized: now 1782025123571040 us; error 124 us; skew 500 ppm
I20260621 06:58:43.571337 11229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-21T06:58:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1730 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:58:43 dist-test-slave-9zdv krb5kdc[11243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:58:43 dist-test-slave-9zdv krb5kdc[11243](info): set up 2 sockets
Jun 21 06:58:43 dist-test-slave-9zdv krb5kdc[11243](info): commencing operation
krb5kdc: starting...
W20260621 06:58:44.740103 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:58:44 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:58:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:58:44.956526 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35783
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:45.272850 11256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:45.273188 11256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:45.273350 11256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:45.280411 11256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:45.280602 11256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:45.280700 11256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:45.280795 11256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:45.280867 11256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:45.290421 11256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:45.291981 11256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:45.293310 11256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:45.304903 11264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:45.304881 11262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:45.304852 11261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:45.305660 11256 server_base.cc:1034] running on GCE node
I20260621 06:58:45.306406 11256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:45.307885 11256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:45.309084 11256 hybrid_clock.cc:648] HybridClock initialized: now 1782025125309035 us; error 44 us; skew 500 ppm
I20260621 06:58:45.309382 11256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:45 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:45.315825 11256 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:45.317349 11256 webserver.cc:460] Webserver started at http://127.2.245.254:35179/ using document root <none> and password file <none>
I20260621 06:58:45.318058 11256 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:45.318197 11256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:45.318519 11256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:45.321547 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "1422811fda35013f58966f31ad2938bd"
server_key_iv: "8a6531e100ece72b5810a913c2aaf205"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.322337 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "1422811fda35013f58966f31ad2938bd"
server_key_iv: "8a6531e100ece72b5810a913c2aaf205"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.327827 11256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:58:45.331694 11271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:45.333081 11256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:45.333264 11256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "1422811fda35013f58966f31ad2938bd"
server_key_iv: "8a6531e100ece72b5810a913c2aaf205"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.333474 11256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:45.359596 11256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:45.363974 11256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:45.364279 11256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:45.382637 11256 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35783
I20260621 06:58:45.382655 11322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35783 every 8 connection(s)
I20260621 06:58:45.384105 11256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:45.387042 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11256
I20260621 06:58:45.387252 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:58:45.388044 3031 external_mini_cluster.cc:1428] Setting key 3e08ab35f01f2b1572bc451b87031297
I20260621 06:58:45.388655 11323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:58:45.403402 11323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Bootstrap starting.
Jun 21 06:58:45 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:45.406662 11323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Neither blocks nor log segments found. Creating new log.
I20260621 06:58:45.407790 11323 log.cc:824] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:45.410579 11323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: No bootstrap required, opened a new log
I20260621 06:58:45.412070 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:45.390200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.2.245.254:35783)
0621 06:58:45.391085 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:45.391097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:45.391142 (+ 45us) server_negotiation.cc:407] Connection header received
0621 06:58:45.393402 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:45.393427 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:45.393886 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:45.394257 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:45.396133 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:45.397425 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:45.399214 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:45.399945 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:45.400092 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:45.404822 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:45.404831 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:45.404847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:45.404892 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:45.408094 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:45.408789 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:45.408799 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:45.408805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:45.408967 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:45.409355 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:45.409360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:45.409362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:45.409881 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:45.410176 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:45.410609 (+ 433us) server_negotiation.cc:299] Negotiation successful
0621 06:58:45.411080 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":393,"threads_started":1}
I20260621 06:58:45.419412 11323 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:45.419809 11323 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:58:45.419919 11323 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe63a39497d46d5ba6e2f81bad6ea4f, State: Initialized, Role: FOLLOWER
I20260621 06:58:45.420462 11323 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:45.420701 11323 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:45.420842 11323 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:45.421008 11323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:58:45.423276 11323 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:45.423805 11323 leader_election.cc:302] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe63a39497d46d5ba6e2f81bad6ea4f; no voters:
I20260621 06:58:45.424580 11323 leader_election.cc:288] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:58:45.424737 11328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:58:45.426657 11323 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:45.426702 11328 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 LEADER]: Becoming Leader. State: Replica: cbe63a39497d46d5ba6e2f81bad6ea4f, State: Running, Role: LEADER
I20260621 06:58:45.427274 11328 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:45.430369 11330 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe63a39497d46d5ba6e2f81bad6ea4f. Latest consensus state: current_term: 1 leader_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } } }
I20260621 06:58:45.430757 11330 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:45.431197 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } } }
I20260621 06:58:45.431545 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:45.433521 11338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:45.438325 11338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:45.446466 11338 catalog_manager.cc:1347] Generated new cluster ID: 68748e0e4d7b4fd9ab85742e8fb15488
I20260621 06:58:45.446583 11338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:45.459462 11338 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:58:45.461366 11338 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:45.481727 11338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Generated new TSK 0
I20260621 06:58:45.482864 11338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:58:45.540499 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:45.874114 11349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:45.874372 11349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:45.874524 11349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:45.881727 11349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:45.881855 11349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:45.882030 11349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:45.892495 11349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:45.894089 11349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:45.895452 11349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:45.910988 11354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:45.911091 11357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:45.911024 11355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:45.912818 11349 server_base.cc:1034] running on GCE node
I20260621 06:58:45.913415 11349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:45.914477 11349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:45.915706 11349 hybrid_clock.cc:648] HybridClock initialized: now 1782025125915640 us; error 58 us; skew 500 ppm
I20260621 06:58:45.916035 11349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:45 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:45.919085 11349 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:58:45.920475 11349 webserver.cc:460] Webserver started at http://127.2.245.193:46699/ using document root <none> and password file <none>
I20260621 06:58:45.921180 11349 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:45.921319 11349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:45.921614 11349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:45.924471 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "5c687b56cd41454f85d577df1d9cafb0"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "9cec84f0134dd25044ac3f9fa926da5c"
server_key_iv: "b967fb03b6414990c9373adbc3b2817a"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.925238 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "5c687b56cd41454f85d577df1d9cafb0"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "9cec84f0134dd25044ac3f9fa926da5c"
server_key_iv: "b967fb03b6414990c9373adbc3b2817a"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.930783 11349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:58:45.934087 11364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:45.935397 11349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:58:45.935557 11349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "5c687b56cd41454f85d577df1d9cafb0"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "9cec84f0134dd25044ac3f9fa926da5c"
server_key_iv: "b967fb03b6414990c9373adbc3b2817a"
server_key_version: "encryptionkey@0"
I20260621 06:58:45.935745 11349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:45.960237 11349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:45.964293 11349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:45.964584 11349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:45.965512 11349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:45.966828 11349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:45.966929 11349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:45.967085 11349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:45.967160 11349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:45.999635 11349 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:37465
I20260621 06:58:45.999686 11476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:37465 every 8 connection(s)
I20260621 06:58:46.000861 11349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:46.001626 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11349
I20260621 06:58:46.001852 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:58:46.002674 3031 external_mini_cluster.cc:1428] Setting key b6c6aeda3967f87a6e8615b5830cf076
Jun 21 06:58:46 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:46.022761 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.003975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:32991 (local address 127.2.245.254:35783)
0621 06:58:46.004458 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:46.004465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:46.005484 (+ 1019us) server_negotiation.cc:407] Connection header received
0621 06:58:46.007370 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:46.007378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:46.007466 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:46.007592 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:46.009068 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.010233 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.012534 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.012942 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.013044 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:46.017072 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:46.017076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:46.017079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:46.017112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:46.019693 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:46.020392 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:46.020397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:46.020399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:46.020510 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:46.021022 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:46.021027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:46.021031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:46.021282 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:46.021429 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:46.022153 (+ 724us) server_negotiation.cc:299] Negotiation successful
0621 06:58:46.022385 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":235,"threads_started":1}
I20260621 06:58:46.023773 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.004415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:32991)
0621 06:58:46.005293 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:46.005350 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:46.007109 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:46.007790 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:46.007797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:46.008339 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:46.008895 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:46.008912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.010387 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.010391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:46.012381 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:46.012405 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.013185 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.013194 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:46.013518 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:46.014215 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:46.014247 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:46.016849 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:46.019830 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:46.019840 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:46.019856 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:46.020265 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:46.020641 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:46.020646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:46.020650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:46.020902 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:46.021401 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:46.021409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:46.021777 (+ 368us) client_negotiation.cc:769] Sending connection context
0621 06:58:46.022038 (+ 261us) client_negotiation.cc:240] Negotiation successful
0621 06:58:46.022561 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":357,"threads_started":1}
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:58:46.025336 11477 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35783
I20260621 06:58:46.025694 11477 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:46.026479 11477 heartbeater.cc:511] Master 127.2.245.254:35783 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 06:58:46.029145 11288 ts_manager.cc:194] Registered new tserver with Master: 5c687b56cd41454f85d577df1d9cafb0 (127.2.245.193:37465)
I20260621 06:58:46.031283 11288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:32991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:58:46.054208 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:46.372438 11483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:46.372766 11483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:46.372947 11483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:46.380033 11483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:46.380187 11483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:46.380364 11483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:46.389982 11483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:46.391237 11483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:46.392525 11483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:46.405589 11488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:46.405671 11489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:46.405630 11491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:46.406541 11483 server_base.cc:1034] running on GCE node
I20260621 06:58:46.407207 11483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:46.408113 11483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:46.409328 11483 hybrid_clock.cc:648] HybridClock initialized: now 1782025126409277 us; error 37 us; skew 500 ppm
I20260621 06:58:46.409652 11483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:46 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025126, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:46.412932 11483 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:58:46.414508 11483 webserver.cc:460] Webserver started at http://127.2.245.194:37763/ using document root <none> and password file <none>
I20260621 06:58:46.415307 11483 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:46.415411 11483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:46.415668 11483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:46.418499 11483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "a7844d85848a49c989deb16963060138"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "44eb40ce3d5b352776a030e46ee88cbf"
server_key_iv: "c2bac7ff7630d0cb7dbb4cfec20c1c72"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.419293 11483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "a7844d85848a49c989deb16963060138"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "44eb40ce3d5b352776a030e46ee88cbf"
server_key_iv: "c2bac7ff7630d0cb7dbb4cfec20c1c72"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.424265 11483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 06:58:46.427769 11498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.429172 11483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:46.429332 11483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "a7844d85848a49c989deb16963060138"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "44eb40ce3d5b352776a030e46ee88cbf"
server_key_iv: "c2bac7ff7630d0cb7dbb4cfec20c1c72"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.429519 11483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:46.453689 11483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:46.457739 11483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:46.458050 11483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:46.459012 11483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:46.460276 11483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:46.460345 11483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.460479 11483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:46.460525 11483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.493494 11483 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42819
I20260621 06:58:46.493582 11610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42819 every 8 connection(s)
I20260621 06:58:46.494776 11483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:46.504314 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11483
I20260621 06:58:46.504472 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:58:46.504956 3031 external_mini_cluster.cc:1428] Setting key 6ec16ae417711f0d5c8a1ace44c2a695
Jun 21 06:58:46 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025126, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:46.515735 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.497431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:54717 (local address 127.2.245.254:35783)
0621 06:58:46.497629 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:46.497633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:46.498891 (+ 1258us) server_negotiation.cc:407] Connection header received
0621 06:58:46.500760 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:46.500766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:46.500861 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:46.500968 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:46.502431 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.503308 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.504927 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.505443 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.505574 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:46.509046 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:46.509054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:46.509061 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:46.509106 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:46.512804 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:46.513555 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:46.513562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:46.513568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:46.513727 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:46.514112 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:46.514117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:46.514120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:46.514387 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:46.514535 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:46.515331 (+ 796us) server_negotiation.cc:299] Negotiation successful
0621 06:58:46.515493 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:58:46.517311 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.497930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:54717)
0621 06:58:46.498600 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:46.498661 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:46.500477 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:46.501202 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:46.501211 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:46.501712 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:46.502262 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:46.502280 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.503467 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.503471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:46.504786 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:46.504798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:46.505690 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:46.505698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:46.505918 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:46.506594 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:46.506627 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:46.508826 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:46.512956 (+ 4130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:46.512963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:46.512975 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:46.513441 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:46.513830 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:46.513832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:46.513834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:46.513999 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:46.514528 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:46.514536 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:46.514855 (+ 319us) client_negotiation.cc:769] Sending connection context
0621 06:58:46.515255 (+ 400us) client_negotiation.cc:240] Negotiation successful
0621 06:58:46.515809 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":264,"threads_started":1}
I20260621 06:58:46.518676 11611 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35783
I20260621 06:58:46.519073 11611 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:46.519770 11611 heartbeater.cc:511] Master 127.2.245.254:35783 requested a full tablet report, sending...
I20260621 06:58:46.521127 11288 ts_manager.cc:194] Registered new tserver with Master: a7844d85848a49c989deb16963060138 (127.2.245.194:42819)
I20260621 06:58:46.521909 11288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:54717
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:58:46.552561 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:46.871690 11616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:46.872018 11616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:46.872172 11616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:46.879498 11616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:46.879639 11616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:46.879801 11616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:46.889418 11616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:46.890676 11616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:46.892004 11616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:46.905879 11621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:46.905915 11624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:46.905942 11622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:46.907089 11616 server_base.cc:1034] running on GCE node
I20260621 06:58:46.907689 11616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:46.908564 11616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:46.909793 11616 hybrid_clock.cc:648] HybridClock initialized: now 1782025126909743 us; error 59 us; skew 500 ppm
I20260621 06:58:46.910082 11616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:46 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025126, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:46.913220 11616 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:58:46.914605 11616 webserver.cc:460] Webserver started at http://127.2.245.195:43313/ using document root <none> and password file <none>
I20260621 06:58:46.915333 11616 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:46.915479 11616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:46.915766 11616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:58:46.918543 11616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "41b3aebdf3274052a1f44e96f9d81304"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "fee733e7de5b739481291cca3e014d35"
server_key_iv: "552bb9f675d51316db88cf42a51c7b25"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.919317 11616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "41b3aebdf3274052a1f44e96f9d81304"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "fee733e7de5b739481291cca3e014d35"
server_key_iv: "552bb9f675d51316db88cf42a51c7b25"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.924288 11616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 06:58:46.927937 11631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.929289 11616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:58:46.929461 11616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "41b3aebdf3274052a1f44e96f9d81304"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "fee733e7de5b739481291cca3e014d35"
server_key_iv: "552bb9f675d51316db88cf42a51c7b25"
server_key_version: "encryptionkey@0"
I20260621 06:58:46.929646 11616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:46.951390 11616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:46.955528 11616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:46.955813 11616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:46.956768 11616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:46.958079 11616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:46.958189 11616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.958361 11616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:46.958428 11616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:46.990774 11616 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:44633
I20260621 06:58:46.990852 11743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:44633 every 8 connection(s)
I20260621 06:58:46.992043 11616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:46.992683 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11616
I20260621 06:58:46.992861 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:58:46.993438 3031 external_mini_cluster.cc:1428] Setting key d4cd19cdf47159beab0336e0142b671f
Jun 21 06:58:47 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025126, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:58:47.012475 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.994814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:42821 (local address 127.2.245.254:35783)
0621 06:58:46.995018 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:46.995022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:46.996581 (+ 1559us) server_negotiation.cc:407] Connection header received
0621 06:58:46.998350 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:46.998354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:46.998418 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:46.998498 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:47.000016 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.001141 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.002682 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.003307 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.003445 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:47.006673 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:47.006678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:47.006681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:47.006711 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:47.009832 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.010490 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.010493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.010496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.010616 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.011040 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.011044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.011046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.011299 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:47.011408 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:47.012069 (+ 661us) server_negotiation.cc:299] Negotiation successful
0621 06:58:47.012227 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 06:58:47.013429 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:46.995469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:42821)
0621 06:58:46.996426 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:46.996480 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:46.998111 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:46.998687 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:46.998695 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:46.999303 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:46.999856 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:46.999874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.001279 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.001283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:47.002530 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:47.002543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.003497 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.003502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:47.003715 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:47.004377 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:47.004397 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:47.006461 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:58:47.009950 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:47.009957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:47.009968 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:47.010390 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:47.010725 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:58:47.010728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:58:47.010730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:58:47.010892 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:58:47.011408 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:58:47.011413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:58:47.011729 (+ 316us) client_negotiation.cc:769] Sending connection context
0621 06:58:47.011972 (+ 243us) client_negotiation.cc:240] Negotiation successful
0621 06:58:47.012328 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":328,"threads_started":1}
I20260621 06:58:47.014707 11744 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35783
I20260621 06:58:47.015120 11744 heartbeater.cc:464] Registering TS with master...
I20260621 06:58:47.015785 11744 heartbeater.cc:511] Master 127.2.245.254:35783 requested a full tablet report, sending...
I20260621 06:58:47.017136 11288 ts_manager.cc:194] Registered new tserver with Master: 41b3aebdf3274052a1f44e96f9d81304 (127.2.245.195:44633)
I20260621 06:58:47.017935 11288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:42821
I20260621 06:58:47.021111 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:58:47.035069 11477 heartbeater.cc:503] Master 127.2.245.254:35783 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
Entry for principal oryx/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab.
WARNING: no policy specified for oryx/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
Entry for principal oryx/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab.
WARNING: no policy specified for oryx/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
Entry for principal oryx/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab.
I20260621 06:58:47.148945 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11349
Jun 21 06:58:47 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:58:47.164337 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:47.150026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.2.245.193:37465)
0621 06:58:47.150682 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:47.150695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:47.150725 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:58:47.150852 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:47.150859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:47.151128 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:47.151336 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:47.152595 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.153573 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.154676 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.155276 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.155362 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:47.159091 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:47.159101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:47.159116 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:47.159146 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:47.161871 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.162547 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.162554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.162556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.162670 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.163036 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.163039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.163053 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.163388 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:47.163615 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:47.163824 (+ 209us) server_negotiation.cc:299] Negotiation successful
0621 06:58:47.164013 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":328,"threads_started":1}
I20260621 06:58:47.165513 11472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:47.249464 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11349
I20260621 06:58:47.266865 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11483
Jun 21 06:58:47 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:58:47.284142 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:47.267830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.2.245.194:42819)
0621 06:58:47.268373 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:47.268383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:47.268410 (+ 27us) server_negotiation.cc:407] Connection header received
0621 06:58:47.268466 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:47.268472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:47.268678 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:47.268808 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:47.270006 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.271101 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.272394 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.273143 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.273291 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:47.277549 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:47.277560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:47.277574 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:47.277606 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:47.281029 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.281792 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.281800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.281802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.281947 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.282356 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.282362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.282366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.282749 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:47.283110 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:47.283460 (+ 350us) server_negotiation.cc:299] Negotiation successful
0621 06:58:47.283771 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":268,"threads_started":1}
I20260621 06:58:47.285423 11606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:47.370393 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11483
I20260621 06:58:47.386299 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11616
Jun 21 06:58:47 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:58:47.402727 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:47.387445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.2.245.195:44633)
0621 06:58:47.388149 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:47.388156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:47.388186 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:58:47.388245 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:47.388251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:47.388455 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:47.388581 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:47.390190 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.391261 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.392488 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:47.393049 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:47.393156 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:47.397578 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:47.397590 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:47.397605 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:47.397639 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:47.400336 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.400960 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.400967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.400969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.401102 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:47.401447 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:47.401450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:47.401452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:47.401785 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:47.402055 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:47.402247 (+ 192us) server_negotiation.cc:299] Negotiation successful
0621 06:58:47.402449 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":399,"threads_started":1}
I20260621 06:58:47.404145 11739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:47.487087 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11616
I20260621 06:58:47.503449 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11256
I20260621 06:58:47.504675 11318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:58:47.607018 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11256
I20260621 06:58:47.627082 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35783
--webserver_interface=127.2.245.254
--webserver_port=35179
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:58:47.959353 11761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:47.959618 11761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:47.959789 11761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:47.966674 11761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:58:47.966804 11761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:47.966893 11761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:47.967013 11761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:58:47.967077 11761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:58:47.976925 11761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=35179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:47.978251 11761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:47.979743 11761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:47.991467 11766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:47.991531 11769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:47.991549 11767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:47.992347 11761 server_base.cc:1034] running on GCE node
I20260621 06:58:47.993151 11761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:47.994549 11761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:47.995791 11761 hybrid_clock.cc:648] HybridClock initialized: now 1782025127995754 us; error 64 us; skew 500 ppm
I20260621 06:58:47.996126 11761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:47 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025127, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:47.999307 11761 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:58:48.000698 11761 webserver.cc:460] Webserver started at http://127.2.245.254:35179/ using document root <none> and password file <none>
I20260621 06:58:48.001413 11761 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:48.001570 11761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:48.007016 11761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:48.009778 11776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.011212 11761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 06:58:48.011395 11761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "1422811fda35013f58966f31ad2938bd"
server_key_iv: "8a6531e100ece72b5810a913c2aaf205"
server_key_version: "encryptionkey@0"
I20260621 06:58:48.012380 11761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:48.026301 11761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:48.030625 11761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:48.030905 11761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:48.048084 11761 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35783
I20260621 06:58:48.048090 11827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35783 every 8 connection(s)
I20260621 06:58:48.049604 11761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:58:48.050168 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11761
I20260621 06:58:48.051148 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:37465
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:35783
--webserver_port=46699
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 06:58:48.056717 11828 sys_catalog.cc:263] Verifying existing consensus state
I20260621 06:58:48.059944 11828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Bootstrap starting.
I20260621 06:58:48.078696 11828 log.cc:824] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Log is configured to *not* fsync() on all Append() calls
I20260621 06:58:48.086903 11828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 06:58:48.087502 11828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Bootstrap complete.
I20260621 06:58:48.096498 11828 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:48.097354 11828 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe63a39497d46d5ba6e2f81bad6ea4f, State: Initialized, Role: FOLLOWER
I20260621 06:58:48.098027 11828 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:48.098287 11828 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:58:48.098425 11828 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:58:48.098594 11828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 1 FOLLOWER]: Advancing to term 2
I20260621 06:58:48.102391 11828 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:48.102888 11828 leader_election.cc:302] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe63a39497d46d5ba6e2f81bad6ea4f; no voters:
I20260621 06:58:48.103611 11828 leader_election.cc:288] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 06:58:48.103767 11832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 2 FOLLOWER]: Leader election won for term 2
I20260621 06:58:48.104907 11832 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [term 2 LEADER]: Becoming Leader. State: Replica: cbe63a39497d46d5ba6e2f81bad6ea4f, State: Running, Role: LEADER
I20260621 06:58:48.105504 11832 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } }
I20260621 06:58:48.105792 11828 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:58:48.108050 11833 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } } }
I20260621 06:58:48.108467 11833 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:48.110116 11837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:58:48.110152 11834 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe63a39497d46d5ba6e2f81bad6ea4f. Latest consensus state: current_term: 2 leader_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe63a39497d46d5ba6e2f81bad6ea4f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35783 } } }
I20260621 06:58:48.110581 11834 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f [sys.catalog]: This master's current role is: LEADER
I20260621 06:58:48.116006 11837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:58:48.117601 11837 catalog_manager.cc:1259] Loaded cluster ID: 68748e0e4d7b4fd9ab85742e8fb15488
I20260621 06:58:48.117748 11837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:58:48.121680 11837 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:58:48.123170 11837 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cbe63a39497d46d5ba6e2f81bad6ea4f: Loaded TSK: 0
I20260621 06:58:48.124610 11837 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 06:58:48.389760 11830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:48.390076 11830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:48.390240 11830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:48.397687 11830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:48.397855 11830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:48.398028 11830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:58:48.408447 11830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.193.keytab
--principal=oryx/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:37465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=46699
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:48.409811 11830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:48.411238 11830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:48.426728 11855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:48.426728 11854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:48.428054 11857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:48.428095 11830 server_base.cc:1034] running on GCE node
I20260621 06:58:48.428934 11830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:48.430078 11830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:48.431341 11830 hybrid_clock.cc:648] HybridClock initialized: now 1782025128431284 us; error 79 us; skew 500 ppm
I20260621 06:58:48.431746 11830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:48 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025128, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:48.435659 11830 init.cc:377] Logged in from keytab as oryx/127.2.245.193@KRBTEST.COM (short username oryx)
I20260621 06:58:48.437350 11830 webserver.cc:460] Webserver started at http://127.2.245.193:46699/ using document root <none> and password file <none>
I20260621 06:58:48.438057 11830 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:48.438215 11830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:48.443811 11830 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260621 06:58:48.446686 11864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.448145 11830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:48.448326 11830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "5c687b56cd41454f85d577df1d9cafb0"
format_stamp: "Formatted at 2026-06-21 06:58:45 on dist-test-slave-9zdv"
server_key: "9cec84f0134dd25044ac3f9fa926da5c"
server_key_iv: "b967fb03b6414990c9373adbc3b2817a"
server_key_version: "encryptionkey@0"
I20260621 06:58:48.449277 11830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:48.477893 11830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:48.482386 11830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:48.482662 11830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:48.483763 11830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:48.485069 11830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:48.485217 11830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.485347 11830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:48.485430 11830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.519176 11830 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:37465
I20260621 06:58:48.519255 11976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:37465 every 8 connection(s)
I20260621 06:58:48.520649 11830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:58:48.523350 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11830
I20260621 06:58:48.524123 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:42819
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:35783
--webserver_port=37763
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 06:58:48 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:48 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:48.850196 11981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:48.850495 11981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:48.850646 11981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:48.858327 11981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:48.858438 11981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:48.858603 11981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:58:48.868438 11981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.194.keytab
--principal=oryx/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:42819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=37763
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:48.869781 11981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:48.871115 11981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:48.888831 11988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:48.888906 11987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:48.889146 11990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:48.890838 11981 server_base.cc:1034] running on GCE node
I20260621 06:58:48.891583 11981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:48.892601 11981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:48.894053 11981 hybrid_clock.cc:648] HybridClock initialized: now 1782025128893975 us; error 84 us; skew 500 ppm
I20260621 06:58:48.894367 11981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:48 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025128, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:48.897717 11981 init.cc:377] Logged in from keytab as oryx/127.2.245.194@KRBTEST.COM (short username oryx)
I20260621 06:58:48.899422 11981 webserver.cc:460] Webserver started at http://127.2.245.194:37763/ using document root <none> and password file <none>
I20260621 06:58:48.900147 11981 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:48.900290 11981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:48.907548 11981 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:58:48.910460 11997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.911972 11981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 06:58:48.912139 11981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "a7844d85848a49c989deb16963060138"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "44eb40ce3d5b352776a030e46ee88cbf"
server_key_iv: "c2bac7ff7630d0cb7dbb4cfec20c1c72"
server_key_version: "encryptionkey@0"
I20260621 06:58:48.913096 11981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:48.927438 11981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:48.931975 11981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:48.932235 11981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:48.933259 11981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:48.934648 11981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:48.934725 11981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.934872 11981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:48.934971 11981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:48.980882 11981 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42819
I20260621 06:58:48.980927 12109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42819 every 8 connection(s)
I20260621 06:58:48.982334 11981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:58:48.987756 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 11981
I20260621 06:58:48.988466 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:44633
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:35783
--webserver_port=43313
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 06:58:48 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:49 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:49.358397 12114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:58:49.358704 12114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:58:49.358860 12114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:58:49.366034 12114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:58:49.366187 12114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:58:49.366371 12114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:58:49.376569 12114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/krb5kdc/oryx_127.2.245.195.keytab
--principal=oryx/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:44633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=43313
--tserver_master_addrs=127.2.245.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:58:49.377944 12114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:58:49.379405 12114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:58:49.397565 12120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:49.397799 12121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:58:49.406693 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:58:49.407310 12114 server_base.cc:1034] running on GCE node
I20260621 06:58:49.408022 12114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:58:49.409103 12114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:58:49.410363 12114 hybrid_clock.cc:648] HybridClock initialized: now 1782025129410295 us; error 67 us; skew 500 ppm
I20260621 06:58:49.410708 12114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:58:49 dist-test-slave-9zdv krb5kdc[11243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025129, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:58:49.414103 12114 init.cc:377] Logged in from keytab as oryx/127.2.245.195@KRBTEST.COM (short username oryx)
I20260621 06:58:49.416610 12114 webserver.cc:460] Webserver started at http://127.2.245.195:43313/ using document root <none> and password file <none>
I20260621 06:58:49.417438 12114 fs_manager.cc:359] Metadata directory not provided
I20260621 06:58:49.417606 12114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:58:49.423713 12114 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260621 06:58:49.426566 12130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:49.428049 12114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 06:58:49.428224 12114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "41b3aebdf3274052a1f44e96f9d81304"
format_stamp: "Formatted at 2026-06-21 06:58:46 on dist-test-slave-9zdv"
server_key: "fee733e7de5b739481291cca3e014d35"
server_key_iv: "552bb9f675d51316db88cf42a51c7b25"
server_key_version: "encryptionkey@0"
I20260621 06:58:49.429215 12114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:58:49.493907 12114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:58:49.502457 12114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:58:49.502887 12114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:58:49.504680 12114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:58:49.507324 12114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:58:49.507510 12114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:49.507688 12114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:58:49.507810 12114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:58:49.566493 12114 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:44633
I20260621 06:58:49.566556 12242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:44633 every 8 connection(s)
I20260621 06:58:49.568147 12114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:58:49.575688 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12114
Jun 21 06:58:49 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:49 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:49.628700 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:49.576646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.2.245.254:35783)
0621 06:58:49.599485 (+ 22839us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:49.599495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:49.599526 (+ 31us) server_negotiation.cc:407] Connection header received
0621 06:58:49.599616 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:49.599624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:49.599733 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:49.599901 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:49.602911 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.604805 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.607738 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.608713 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.613856 (+ 5143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:49.614305 (+ 449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:58:49.614320 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:58:49.614353 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:58:49.614420 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:58:49.620734 (+ 6314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:49.622076 (+ 1342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:49.622090 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:49.622105 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:49.622404 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:58:49.623078 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:58:49.623086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:58:49.623092 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:58:49.624006 (+ 914us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:58:49.625984 (+ 1978us) server_negotiation.cc:1035] Waiting for connection context
0621 06:58:49.626382 (+ 398us) server_negotiation.cc:299] Negotiation successful
0621 06:58:49.627090 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22644,"thread_start_us":6478,"threads_started":1}
W20260621 06:58:50.030889 11979 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.496s user 0.004s sys 0.010s
W20260621 06:58:50.033288 11980 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:48.523758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:45673 (local address 127.2.245.254:35783)
0621 06:58:48.527477 (+ 3719us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:48.527488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:48.527525 (+ 37us) server_negotiation.cc:407] Connection header received
0621 06:58:48.527828 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:48.527840 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:48.528156 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:48.528522 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:48.530024 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.531469 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.532975 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.533609 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.533831 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.032741 (+1498910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:45673: BlockingRecv error: recv got EOF from 127.2.245.193:45673 (error 108)
Metrics: {"server-negotiator.queue_time_us":3501,"thread_start_us":3321,"threads_started":1}
I20260621 06:58:50.035429 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:48.523755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:45673)
0621 06:58:48.524573 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:48.524628 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:48.526269 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:48.528667 (+ 2398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:48.528679 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:48.529321 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:48.529830 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:48.529847 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.531623 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.531627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:48.532822 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:48.532834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.533860 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.533866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:48.534142 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:48.534827 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:48.534848 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.033560 (+1498712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":317,"threads_started":1}
W20260621 06:58:50.037482 11977 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:35783 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.245.254:35783: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:50.057910 11980 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.038533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:32941 (local address 127.2.245.254:35783)
0621 06:58:50.038826 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.038836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.039325 (+ 489us) server_negotiation.cc:407] Connection header received
0621 06:58:50.039430 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.039439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.039575 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.039730 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.041022 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.042728 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.045467 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.046444 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.046637 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.057493 (+ 10856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:32941: BlockingRecv error: recv got EOF from 127.2.245.193:32941 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260621 06:58:50.058962 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.038412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:32941)
0621 06:58:50.038856 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.038885 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.039097 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.039807 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.039814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.040504 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.040819 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.040834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.042972 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.042981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.045260 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.045281 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.046705 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.046714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.047004 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.050727 (+ 3723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.050764 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.058494 (+ 7730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230}
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:50.084821 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.060169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:59981)
0621 06:58:50.068135 (+ 7966us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.068165 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.068377 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.068926 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.068934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.069810 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.070240 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.070257 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.072373 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.072383 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.074754 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.074775 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.076667 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.076676 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.076953 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.077941 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.077971 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.084326 (+ 6355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7750,"thread_start_us":7452,"threads_started":1}
W20260621 06:58:50.085026 11980 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.060260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:59981 (local address 127.2.245.254:35783)
0621 06:58:50.060509 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.060518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.068315 (+ 7797us) server_negotiation.cc:407] Connection header received
0621 06:58:50.068557 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.068565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.068705 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.068863 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.070452 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.072180 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.075036 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.076072 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.076209 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.084671 (+ 8462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:59981: BlockingRecv error: recv got EOF from 127.2.245.193:59981 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260621 06:58:50.085515 11977 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 06:58:50.418340 12112 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.418s user 0.004s sys 0.013s
W20260621 06:58:50.419997 12113 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:48.987125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:50267 (local address 127.2.245.254:35783)
0621 06:58:48.991433 (+ 4308us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:48.991441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:48.991463 (+ 22us) server_negotiation.cc:407] Connection header received
0621 06:58:48.991526 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:48.991533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:48.991617 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:48.991729 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:48.994816 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.995643 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.997543 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.998564 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.998751 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.419751 (+1421000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:50267: BlockingRecv error: recv got EOF from 127.2.245.194:50267 (error 108)
Metrics: {"server-negotiator.queue_time_us":4164,"thread_start_us":4055,"threads_started":1}
I20260621 06:58:50.421233 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:48.986378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:50267)
0621 06:58:48.987350 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:48.987745 (+ 395us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:48.990482 (+ 2737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:48.992173 (+ 1691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:48.992192 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:48.993298 (+ 1106us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:48.994519 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:48.994573 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.995816 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.995820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:48.997324 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:48.997343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:48.998868 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:48.998877 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:48.999238 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:49.000200 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:49.000240 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.419622 (+1419382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":818,"threads_started":1}
W20260621 06:58:50.422224 12110 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:35783 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.245.254:35783: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:50.438705 12113 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.423131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59947 (local address 127.2.245.254:35783)
0621 06:58:50.423294 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.423298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.423314 (+ 16us) server_negotiation.cc:407] Connection header received
0621 06:58:50.423530 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.423533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.423591 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.423679 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.424893 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.425639 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.428268 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.428704 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.428810 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.438527 (+ 9717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:59947: BlockingRecv error: recv got EOF from 127.2.245.194:59947 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 06:58:50.438789 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.422906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:59947)
0621 06:58:50.423206 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.423234 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.423389 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.423766 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.423774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.424423 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.424724 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.424740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.425814 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.425823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.428071 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.428091 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.428945 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.428953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.429207 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.432878 (+ 3671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.432933 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.438363 (+ 5430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:50.449965 12113 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.440158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:54441 (local address 127.2.245.254:35783)
0621 06:58:50.440334 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.440338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.440351 (+ 13us) server_negotiation.cc:407] Connection header received
0621 06:58:50.440390 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.440393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.440460 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.440567 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.441220 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.441915 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.443297 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.444274 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.444459 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.449752 (+ 5293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:54441: BlockingRecv error: recv got EOF from 127.2.245.194:54441 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 06:58:50.450023 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.439672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:54441)
0621 06:58:50.439831 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.439850 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.439979 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.440601 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.440606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.440926 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.441079 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.441087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.442058 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.442063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.443139 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.443155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.444453 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.444458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.444583 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.445197 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.445214 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.449598 (+ 4384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260621 06:58:50.450495 12110 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 06:58:50.761833 12245 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.144s user 0.004s sys 0.015s
W20260621 06:58:50.762917 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:49.573065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:56427 (local address 127.2.245.254:35783)
0621 06:58:49.583517 (+ 10452us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:49.583528 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:49.600978 (+ 17450us) server_negotiation.cc:407] Connection header received
0621 06:58:49.605027 (+ 4049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:49.605037 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:49.605151 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:49.605255 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:49.609032 (+ 3777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.610885 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.614615 (+ 3730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.615644 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.615764 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.762703 (+1146939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:56427: BlockingRecv error: recv got EOF from 127.2.245.195:56427 (error 108)
Metrics: {"server-negotiator.queue_time_us":10234,"thread_start_us":26082,"threads_started":1}
I20260621 06:58:50.763485 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:49.572384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:56427)
0621 06:58:49.600807 (+ 28423us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:49.600868 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:49.604533 (+ 3665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:49.606185 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:49.606205 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:49.607478 (+ 1273us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:49.608714 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:49.608763 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.611214 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.611223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:49.614362 (+ 3139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:49.614384 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:49.616280 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:49.616290 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:49.616712 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:49.617696 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:49.617739 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.762522 (+1144783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28089,"thread_start_us":27829,"threads_started":1}
W20260621 06:58:50.764381 12243 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:35783 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.245.254:35783: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:50.773404 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.765051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:34849)
0621 06:58:50.765262 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.765279 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.765392 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.765755 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.765760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.766121 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.766286 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.766294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.767361 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.767366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.768291 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.768303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.769028 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.769033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.769178 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.769721 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.769737 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.773119 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260621 06:58:50.773550 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.765293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:34849 (local address 127.2.245.254:35783)
0621 06:58:50.765494 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.765499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.765514 (+ 15us) server_negotiation.cc:407] Connection header received
0621 06:58:50.765559 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.765562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.765631 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.765726 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.766420 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.767219 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.768429 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.768848 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.768985 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.773307 (+ 4322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:34849: BlockingRecv error: recv got EOF from 127.2.245.195:34849 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:50 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:50.782711 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:50.774181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:51215)
0621 06:58:50.774314 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:50.774329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:50.774439 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:50.774795 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:50.774800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:50.775169 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:50.775314 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.775322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.776336 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.776341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.777810 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:50.777824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.778665 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.778671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:50.778875 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.779505 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:50.779522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:50.782531 (+ 3009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 06:58:50.783052 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:50.774269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:51215 (local address 127.2.245.254:35783)
0621 06:58:50.774564 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:50.774568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:50.774580 (+ 12us) server_negotiation.cc:407] Connection header received
0621 06:58:50.774616 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:50.774619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:50.774667 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:50.774766 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:50.775490 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.776199 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.777928 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:50.778517 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:50.778653 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:50.782727 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:51215: BlockingRecv error: recv got EOF from 127.2.245.195:51215 (error 108)
Metrics: {"server-negotiator.queue_time_us":173}
W20260621 06:58:50.783075 12243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:51.094693 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:51.086094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:46875)
0621 06:58:51.086658 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:51.086678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:51.086819 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:51.087257 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:51.087262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:51.087665 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:51.087813 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.087821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.088812 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.088818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.089742 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.089753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.090477 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.090493 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.090666 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.091247 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:51.091264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:51.094464 (+ 3200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":299,"threads_started":1}
W20260621 06:58:51.094849 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:51.086184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:46875 (local address 127.2.245.254:35783)
0621 06:58:51.086424 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:51.086432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:51.086767 (+ 335us) server_negotiation.cc:407] Connection header received
0621 06:58:51.086974 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:51.086997 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:51.087111 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:51.087222 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:51.087955 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.088664 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.089895 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.090311 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.090442 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.094647 (+ 4205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:46875: BlockingRecv error: recv got EOF from 127.2.245.193:46875 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:51.460592 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:51.451202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:55537)
0621 06:58:51.451746 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:51.451762 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:51.451923 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:51.452288 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:51.452292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:51.452777 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:51.452917 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.452926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.453950 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.453956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.454914 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.454925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.455730 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.455736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.455910 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.456579 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:51.456602 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:51.460354 (+ 3752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":320,"threads_started":1}
W20260621 06:58:51.460803 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:51.451258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:55537 (local address 127.2.245.254:35783)
0621 06:58:51.451499 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:51.451516 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:51.451853 (+ 337us) server_negotiation.cc:407] Connection header received
0621 06:58:51.452091 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:51.452095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:51.452155 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:51.452259 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:51.453070 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.453812 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.455115 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.455547 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.455676 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.460557 (+ 4881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:55537: BlockingRecv error: recv got EOF from 127.2.245.194:55537 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:51 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:51.793324 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:51.783756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:57387)
0621 06:58:51.784438 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:51.784453 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:51.784612 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:51.784943 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:51.784947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:51.785390 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:51.785539 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.785550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.786564 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.786571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.787562 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:51.787573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.788322 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.788328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:51.788509 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.789186 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:51.789212 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:51.793013 (+ 3801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":424,"threads_started":1}
W20260621 06:58:51.793576 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:51.783857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:57387 (local address 127.2.245.254:35783)
0621 06:58:51.784212 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:51.784220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:51.784547 (+ 327us) server_negotiation.cc:407] Connection header received
0621 06:58:51.784733 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:51.784740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:51.784805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:51.784908 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:51.785688 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.786402 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.787704 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:51.788137 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:51.788279 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:51.793287 (+ 5008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:57387: BlockingRecv error: recv got EOF from 127.2.245.195:57387 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:52.104319 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:52.095649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:56419)
0621 06:58:52.096096 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:52.096113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:52.096226 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:52.096521 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:52.096525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:52.096881 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:52.097027 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.097034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.098019 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.098023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.099194 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.099206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.099895 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.099900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.100109 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.100662 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:52.100678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:52.103958 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":230,"threads_started":1}
W20260621 06:58:52.104426 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:52.095729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:56419 (local address 127.2.245.254:35783)
0621 06:58:52.095980 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:52.095986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:52.096192 (+ 206us) server_negotiation.cc:407] Connection header received
0621 06:58:52.096324 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:52.096328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:52.096390 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:52.096468 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:52.097197 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.097897 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.099334 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.099754 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.099887 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.104170 (+ 4283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:56419: BlockingRecv error: recv got EOF from 127.2.245.193:56419 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:52.470296 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:52.461531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:48053)
0621 06:58:52.462041 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:52.462057 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:52.462171 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:52.462478 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:52.462481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:52.462892 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:52.463109 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.463119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.464099 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.464103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.464993 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.465004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.465659 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.465663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.465781 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.466354 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:52.466384 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:52.470066 (+ 3682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":286,"threads_started":1}
W20260621 06:58:52.470361 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:52.461563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:48053 (local address 127.2.245.254:35783)
0621 06:58:52.461757 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:52.461761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:52.462134 (+ 373us) server_negotiation.cc:407] Connection header received
0621 06:58:52.462269 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:52.462272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:52.462329 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:52.462408 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:52.463237 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.463976 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.465116 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.465516 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.465614 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.470204 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:48053: BlockingRecv error: recv got EOF from 127.2.245.194:48053 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:52 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:52.803190 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:52.794385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:60665)
0621 06:58:52.794900 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:52.794916 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:52.795065 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:52.795511 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:52.795516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:52.795871 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:52.796042 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.796049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.797094 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.797098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.798008 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:52.798017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.798695 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.798698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:52.798814 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.799460 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:52.799477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:52.802894 (+ 3417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":294,"threads_started":1}
W20260621 06:58:52.803369 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:52.794554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:60665 (local address 127.2.245.254:35783)
0621 06:58:52.794776 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:52.794780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:52.795027 (+ 247us) server_negotiation.cc:407] Connection header received
0621 06:58:52.795195 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:52.795202 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:52.795319 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:52.795460 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:52.796196 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.796978 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.798162 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:52.798566 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:52.798664 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:52.803142 (+ 4478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:60665: BlockingRecv error: recv got EOF from 127.2.245.195:60665 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:53.113811 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:53.105324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:41767)
0621 06:58:53.105862 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:53.105888 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:53.105996 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:53.106296 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:53.106299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:53.106650 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:53.106824 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.106832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.107891 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.107894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.108794 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.108823 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.109491 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.109496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.109621 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.110179 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:53.110195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:53.113507 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":271,"threads_started":1}
W20260621 06:58:53.113926 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:53.105385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41767 (local address 127.2.245.254:35783)
0621 06:58:53.105541 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:53.105546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:53.105957 (+ 411us) server_negotiation.cc:407] Connection header received
0621 06:58:53.106091 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:53.106095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:53.106166 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:53.106250 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:53.107044 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.107766 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.108940 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.109363 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.109458 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.113651 (+ 4193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:41767: BlockingRecv error: recv got EOF from 127.2.245.193:41767 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
W20260621 06:58:53.481866 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:53.471658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:53965 (local address 127.2.245.254:35783)
0621 06:58:53.471883 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:53.471891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:53.472222 (+ 331us) server_negotiation.cc:407] Connection header received
0621 06:58:53.472420 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:53.472426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:53.472495 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:53.472599 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:53.473317 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.474039 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.475523 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.476307 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.476443 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.481635 (+ 5192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:53965: BlockingRecv error: recv got EOF from 127.2.245.194:53965 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:58:53.481865 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:53.471603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:53965)
0621 06:58:53.472104 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:53.472119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:53.472290 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:53.472639 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:53.472646 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:53.472994 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:53.473166 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.473174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.474178 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.474186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.475353 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.475369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.476452 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.476457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.476713 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.477427 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:53.477447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:53.481435 (+ 3988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":282,"threads_started":1}
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:53 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:53.813403 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:53.804126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:40529)
0621 06:58:53.804736 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:53.804752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:53.804913 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:53.805264 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:53.805268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:53.805724 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:53.805884 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.805892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.807024 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.807031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.808288 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:53.808300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.809001 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.809006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:53.809184 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.809733 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:53.809749 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:53.813064 (+ 3315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":346,"threads_started":1}
W20260621 06:58:53.813506 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:53.804229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:40529 (local address 127.2.245.254:35783)
0621 06:58:53.804563 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:53.804569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:53.804835 (+ 266us) server_negotiation.cc:407] Connection header received
0621 06:58:53.805050 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:53.805054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:53.805130 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:53.805233 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:53.806028 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.806864 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.808427 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:53.808852 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:53.808979 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:53.813268 (+ 4289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:40529: BlockingRecv error: recv got EOF from 127.2.245.195:40529 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:54.124269 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:54.115277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:56035)
0621 06:58:54.115832 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:54.115853 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:54.115972 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:54.116349 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:54.116356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:54.116933 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:54.117146 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.117157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.118366 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.118370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.119430 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.119442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.120071 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.120074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.120185 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.120761 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:54.120777 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:54.124014 (+ 3237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":398,"threads_started":1}
W20260621 06:58:54.124629 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:54.115469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:56035 (local address 127.2.245.254:35783)
0621 06:58:54.115658 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:54.115664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:54.115950 (+ 286us) server_negotiation.cc:407] Connection header received
0621 06:58:54.116084 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:54.116092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:54.116186 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:54.116284 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:54.117388 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.118204 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.119552 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.119953 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.120046 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.124243 (+ 4197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:56035: BlockingRecv error: recv got EOF from 127.2.245.193:56035 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:54.492383 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:54.482914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:34447)
0621 06:58:54.483492 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:54.483508 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:54.483620 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:54.483988 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:54.483992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:54.484366 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:54.484527 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.484535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.485556 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.485560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.486580 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.486592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.487532 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.487537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.487719 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.488564 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:54.488587 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:54.492146 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":281,"threads_started":1}
W20260621 06:58:54.492542 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:54.483100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:34447 (local address 127.2.245.254:35783)
0621 06:58:54.483279 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:54.483284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:54.483593 (+ 309us) server_negotiation.cc:407] Connection header received
0621 06:58:54.483737 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:54.483740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:54.483802 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:54.483890 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:54.484677 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.485422 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.486726 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.487365 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.487486 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.492335 (+ 4849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:34447: BlockingRecv error: recv got EOF from 127.2.245.194:34447 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:54 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:54.825129 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:54.814540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:45393)
0621 06:58:54.815316 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:54.815332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:54.815655 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:54.816022 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:54.816025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:54.816440 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:54.816612 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.816621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.818309 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.818313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.819361 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:54.819372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.820413 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.820417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:54.820551 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.821172 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:54.821189 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:54.824898 (+ 3709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":298,"threads_started":1}
W20260621 06:58:54.825579 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:54.814708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:45393 (local address 127.2.245.254:35783)
0621 06:58:54.815225 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:54.815232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:54.815404 (+ 172us) server_negotiation.cc:407] Connection header received
0621 06:58:54.815780 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:54.815787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:54.815894 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:54.816005 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:54.816778 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.818143 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.819491 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:54.820268 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:54.820424 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:54.825236 (+ 4812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:45393: BlockingRecv error: recv got EOF from 127.2.245.195:45393 (error 108)
Metrics: {"server-negotiator.queue_time_us":385}
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:55.134202 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:55.125302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:47503)
0621 06:58:55.125895 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:55.125916 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:55.126071 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:55.126408 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:55.126412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:55.126823 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:55.126981 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.126989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.127958 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.127965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.128852 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.128862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.129533 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.129539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.129718 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.130369 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:55.130385 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:55.133884 (+ 3499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":318,"threads_started":1}
W20260621 06:58:55.134379 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:55.125407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:47503 (local address 127.2.245.254:35783)
0621 06:58:55.125612 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:55.125621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:55.125985 (+ 364us) server_negotiation.cc:407] Connection header received
0621 06:58:55.126212 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:55.126216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:55.126275 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:55.126378 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:55.127117 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.127826 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.128984 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.129346 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.129501 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.134110 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:47503: BlockingRecv error: recv got EOF from 127.2.245.193:47503 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:55.505893 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:55.493234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:59457)
0621 06:58:55.493727 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:55.493742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:55.493878 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:55.494185 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:55.494188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:55.494596 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:55.494751 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.494759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.495826 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.495830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.496793 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.496804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.497524 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.497528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.497650 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.501889 (+ 4239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:55.501908 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:55.505606 (+ 3698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":275,"threads_started":1}
W20260621 06:58:55.506232 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:55.493421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59457 (local address 127.2.245.254:35783)
0621 06:58:55.493582 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:55.493587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:55.493865 (+ 278us) server_negotiation.cc:407] Connection header received
0621 06:58:55.493989 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:55.493992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:55.494056 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:55.494146 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:55.494899 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.495672 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.496958 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.497398 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.497496 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.505947 (+ 8451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:59457: BlockingRecv error: recv got EOF from 127.2.245.194:59457 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:55 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:55.835803 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:55.826285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:46719)
0621 06:58:55.826825 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:55.826842 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:55.827023 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:55.827346 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:55.827349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:55.827738 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:55.827902 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.827910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.829196 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.829201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.830278 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:55.830297 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.831006 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.831011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:55.831136 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.831714 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:55.831742 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:55.835445 (+ 3703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":311,"threads_started":1}
W20260621 06:58:55.835906 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:55.826373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:46719 (local address 127.2.245.254:35783)
0621 06:58:55.826584 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:55.826589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:55.826928 (+ 339us) server_negotiation.cc:407] Connection header received
0621 06:58:55.827126 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:55.827135 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:55.827197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:55.827289 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:55.828036 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.829040 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.830403 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:55.830844 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:55.830964 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:55.835635 (+ 4671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:46719: BlockingRecv error: recv got EOF from 127.2.245.195:46719 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:56.144694 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:56.135389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:33555)
0621 06:58:56.135901 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:56.135925 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:56.136044 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:56.136364 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:56.136367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:56.136799 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:56.136953 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.136961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.137941 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.137945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.138855 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.138866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.139832 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.139838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.140023 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.140727 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:56.140748 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:56.144299 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":290,"threads_started":1}
W20260621 06:58:56.144809 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:56.135473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:33555 (local address 127.2.245.254:35783)
0621 06:58:56.135674 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:56.135679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:56.136002 (+ 323us) server_negotiation.cc:407] Connection header received
0621 06:58:56.136152 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:56.136157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:56.136227 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:56.136322 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:56.137073 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.137776 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.139144 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.139671 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.139812 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.144546 (+ 4734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:33555: BlockingRecv error: recv got EOF from 127.2.245.193:33555 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:56.515779 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:56.506873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:59959)
0621 06:58:56.507375 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:56.507390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:56.507547 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:56.507900 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:56.507905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:56.508327 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:56.508463 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.508471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.509525 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.509532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.510415 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.510425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.511111 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.511116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.511275 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.511873 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:56.511890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:56.515554 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
W20260621 06:58:56.515976 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:56.506916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:59959 (local address 127.2.245.254:35783)
0621 06:58:56.507138 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:56.507147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:56.507475 (+ 328us) server_negotiation.cc:407] Connection header received
0621 06:58:56.507674 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:56.507679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:56.507743 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:56.507873 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:56.508616 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.509393 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.510540 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.510925 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.511081 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.515737 (+ 4656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:59959: BlockingRecv error: recv got EOF from 127.2.245.194:59959 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:56 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:56.846055 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:56.836982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:47689)
0621 06:58:56.837657 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:56.837677 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:56.837821 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:56.838130 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:56.838133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:56.838507 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:56.838670 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.838678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.839645 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.839661 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.840571 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:56.840583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.841223 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.841227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:56.841370 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.842085 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:56.842122 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:56.845821 (+ 3699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":371,"threads_started":1}
W20260621 06:58:56.846364 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:56.837036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:47689 (local address 127.2.245.254:35783)
0621 06:58:56.837210 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:56.837218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:56.837763 (+ 545us) server_negotiation.cc:407] Connection header received
0621 06:58:56.837933 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:56.837937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:56.837995 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:56.838079 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:56.838783 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.839511 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.840705 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:56.841080 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:56.841168 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:56.846072 (+ 4904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:47689: BlockingRecv error: recv got EOF from 127.2.245.195:47689 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:57.155362 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:57.145833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:50911)
0621 06:58:57.146354 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:57.146381 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:57.146499 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:57.146798 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:57.146801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:57.147199 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:57.147399 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.147407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.148437 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.148441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.149603 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.149615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.150600 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.150607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.150796 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.151503 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:57.151520 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:57.155002 (+ 3482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":245,"threads_started":1}
W20260621 06:58:57.155475 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:57.145960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:50911 (local address 127.2.245.254:35783)
0621 06:58:57.146209 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:57.146215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:57.146464 (+ 249us) server_negotiation.cc:407] Connection header received
0621 06:58:57.146601 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:57.146605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:57.146668 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:57.146751 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:57.147555 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.148320 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.149766 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.150435 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.150567 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.155230 (+ 4663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:50911: BlockingRecv error: recv got EOF from 127.2.245.193:50911 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:57.526484 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:57.516863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:57503)
0621 06:58:57.517444 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:57.517461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:57.517604 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:57.517979 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:57.517982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:57.518390 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:57.518570 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.518579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.519684 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.519690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.520636 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.520648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.521423 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.521430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.521592 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.522274 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:57.522292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:57.526119 (+ 3827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":226,"threads_started":1}
W20260621 06:58:57.526635 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:57.516938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:57503 (local address 127.2.245.254:35783)
0621 06:58:57.517148 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:57.517157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:57.517554 (+ 397us) server_negotiation.cc:407] Connection header received
0621 06:58:57.517732 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:57.517735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:57.517796 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:57.517949 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:57.518707 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.519484 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.520790 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.521262 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.521402 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.526366 (+ 4964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:57503: BlockingRecv error: recv got EOF from 127.2.245.194:57503 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:57 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:57.856721 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:57.847409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:35369)
0621 06:58:57.848021 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:57.848037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:57.848191 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:57.848525 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:57.848529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:57.848968 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:57.849152 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.849162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.850277 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.850282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.851248 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:57.851260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.852058 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.852063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:57.852232 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.852878 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:57.852913 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:57.856502 (+ 3589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":277,"threads_started":1}
W20260621 06:58:57.856990 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:57.847528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:35369 (local address 127.2.245.254:35783)
0621 06:58:57.847751 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:57.847758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:57.848122 (+ 364us) server_negotiation.cc:407] Connection header received
0621 06:58:57.848323 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:57.848327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:57.848395 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:57.848497 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:57.849300 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.850127 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.851402 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:57.851914 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:57.852045 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:57.856706 (+ 4661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:35369: BlockingRecv error: recv got EOF from 127.2.245.195:35369 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:58.165884 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:58.156438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:49051)
0621 06:58:58.157038 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:58.157056 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:58.157170 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:58.157513 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:58.157518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:58.158171 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:58.158416 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.158430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.159686 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.159692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.160650 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.160660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.161378 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.161386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.161588 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.162208 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:58.162225 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:58.165530 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":303,"threads_started":1}
W20260621 06:58:58.165995 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:58.156607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:49051 (local address 127.2.245.254:35783)
0621 06:58:58.156901 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:58.156924 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:58.157140 (+ 216us) server_negotiation.cc:407] Connection header received
0621 06:58:58.157274 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:58.157278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:58.157339 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:58.157482 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:58.158552 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.159536 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.160796 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.161198 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.161321 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.165748 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:49051: BlockingRecv error: recv got EOF from 127.2.245.193:49051 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:58.536916 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:58.527521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:48817)
0621 06:58:58.527978 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:58.527994 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:58.528106 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:58.528404 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:58.528407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:58.528780 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:58.528993 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.529001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.530117 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.530120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.531151 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.531163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.532196 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.532201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.532310 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.533104 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:58.533127 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:58.536651 (+ 3524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
W20260621 06:58:58.537106 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:58.527657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:48817 (local address 127.2.245.254:35783)
0621 06:58:58.527882 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:58.527889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:58.528069 (+ 180us) server_negotiation.cc:407] Connection header received
0621 06:58:58.528204 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:58.528207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:58.528283 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:58.528363 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:58.529113 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.529999 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.531280 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.532000 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.532157 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.536862 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:48817: BlockingRecv error: recv got EOF from 127.2.245.194:48817 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:58 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:58.871902 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:58.857943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:40005)
0621 06:58:58.858519 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:58.858541 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:58.858698 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:58.859068 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:58.859072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:58.859539 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:58.859727 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.859735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.860798 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.860805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.861736 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:58.861747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.862471 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.862478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:58.862635 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.867675 (+ 5040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:58.867694 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:58.871661 (+ 3967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":271,"threads_started":1}
W20260621 06:58:58.872112 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:58.858033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:40005 (local address 127.2.245.254:35783)
0621 06:58:58.858280 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:58.858288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:58.858629 (+ 341us) server_negotiation.cc:407] Connection header received
0621 06:58:58.858819 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:58.858825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:58.858894 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:58.859037 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:58.859882 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.860662 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.861879 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:58.862286 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:58.862426 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:58.871866 (+ 9440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:40005: BlockingRecv error: recv got EOF from 127.2.245.195:40005 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:59.178592 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:59.167103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:59407)
0621 06:58:59.167819 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:59.167845 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:59.168044 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:59.168594 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:59.168600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:59.169206 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:59.169421 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.169433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.170865 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.170872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.171852 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.171864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.173068 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.173075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.173311 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.174058 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:59.174076 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:59.178225 (+ 4149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":482,"threads_started":1}
W20260621 06:58:59.178666 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:59.167193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:59407 (local address 127.2.245.254:35783)
0621 06:58:59.167424 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:59.167430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:59.167935 (+ 505us) server_negotiation.cc:407] Connection header received
0621 06:58:59.168180 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:59.168185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:59.168286 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:59.168386 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:59.169569 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.170688 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.172007 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.172810 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.172938 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.178425 (+ 5487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:59407: BlockingRecv error: recv got EOF from 127.2.245.193:59407 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:59.546295 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:59.538092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:45057)
0621 06:58:59.538656 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:59.538672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:59.538780 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:59.539110 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:59.539113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:59.539490 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:59.539626 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.539634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.540602 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.540605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.541456 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.541465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.542072 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.542075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.542206 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.542773 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:59.542790 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:59.546072 (+ 3282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":341,"threads_started":1}
W20260621 06:58:59.546459 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:59.538130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:45057 (local address 127.2.245.254:35783)
0621 06:58:59.538356 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:59.538366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:59.538778 (+ 412us) server_negotiation.cc:407] Connection header received
0621 06:58:59.538885 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:59.538889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:59.538967 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:59.539045 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:59.539749 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.540488 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.541567 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.541937 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.542045 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.546233 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:45057: BlockingRecv error: recv got EOF from 127.2.245.194:45057 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:58:59 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:58:59.881479 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:58:59.872831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:36479)
0621 06:58:59.873384 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0621 06:58:59.873400 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:58:59.873504 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:58:59.873806 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:58:59.873809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:58:59.874246 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:58:59.874383 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.874391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.875439 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.875443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.876296 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:58:59.876306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.876949 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.876953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:58:59.877120 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.877789 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:58:59.877806 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:58:59.881242 (+ 3436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":326,"threads_started":1}
W20260621 06:58:59.881619 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:58:59.872940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:36479 (local address 127.2.245.254:35783)
0621 06:58:59.873125 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:58:59.873129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:58:59.873482 (+ 353us) server_negotiation.cc:407] Connection header received
0621 06:58:59.873597 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:58:59.873601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:58:59.873659 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:58:59.873760 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:58:59.874523 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.875331 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.876438 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:58:59.876827 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:58:59.876922 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:58:59.881419 (+ 4497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:36479: BlockingRecv error: recv got EOF from 127.2.245.195:36479 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:00.188643 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:00.179812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:37043)
0621 06:59:00.180241 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:00.180258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:00.180375 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:00.180672 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:00.180675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:00.181047 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:00.181194 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.181201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.182213 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.182217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.183394 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.183410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.184155 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.184160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.184336 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.184900 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:00.184917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:00.188281 (+ 3364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":242,"threads_started":1}
W20260621 06:59:00.188756 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:00.179865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:37043 (local address 127.2.245.254:35783)
0621 06:59:00.180102 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:00.180109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:00.180341 (+ 232us) server_negotiation.cc:407] Connection header received
0621 06:59:00.180468 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:00.180472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:00.180530 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:00.180612 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:00.181339 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.182078 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.183523 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.183972 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.184144 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.188486 (+ 4342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:37043: BlockingRecv error: recv got EOF from 127.2.245.193:37043 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:00.560220 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:00.547327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:45255)
0621 06:59:00.547851 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:00.547868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:00.548030 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:00.548370 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:00.548374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:00.548753 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:00.548887 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.548895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.549919 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.549938 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.551079 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.551091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.551752 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.551757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.551922 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.556568 (+ 4646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:00.556589 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:00.559952 (+ 3363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":303,"threads_started":1}
W20260621 06:59:00.560410 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:00.547390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:45255 (local address 127.2.245.254:35783)
0621 06:59:00.547654 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:00.547662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:00.547958 (+ 296us) server_negotiation.cc:407] Connection header received
0621 06:59:00.548178 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:00.548183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:00.548242 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:00.548343 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:00.549058 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.549756 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.551218 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.551591 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.551716 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.560155 (+ 8439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:45255: BlockingRecv error: recv got EOF from 127.2.245.194:45255 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:00 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:00.891757 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:00.882569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:33847)
0621 06:59:00.883133 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:00.883149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:00.883260 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:00.883601 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:00.883604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:00.884026 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:00.884207 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.884216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.885268 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.885272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.886150 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:00.886160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.886827 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.886844 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:00.886993 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.887648 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:00.887666 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:00.891394 (+ 3728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":284,"threads_started":1}
W20260621 06:59:00.891842 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:00.882738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:33847 (local address 127.2.245.254:35783)
0621 06:59:00.882906 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:00.882911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:00.883223 (+ 312us) server_negotiation.cc:407] Connection header received
0621 06:59:00.883355 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:00.883358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:00.883418 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:00.883500 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:00.884352 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.885133 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.886266 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:00.886658 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:00.886772 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:00.891576 (+ 4804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:33847: BlockingRecv error: recv got EOF from 127.2.245.195:33847 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:01.198849 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:01.189787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:33575)
0621 06:59:01.190297 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:01.190322 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:01.190433 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:01.190794 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:01.190798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:01.191417 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:01.191595 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.191603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.192639 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.192645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.193595 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.193607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.194358 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.194363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.194532 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.195204 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:01.195221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:01.198492 (+ 3271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":299,"threads_started":1}
W20260621 06:59:01.198926 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:01.189924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:33575 (local address 127.2.245.254:35783)
0621 06:59:01.190107 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:01.190111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:01.190407 (+ 296us) server_negotiation.cc:407] Connection header received
0621 06:59:01.190574 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:01.190580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:01.190650 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:01.190758 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:01.191748 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.192485 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.193757 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.194201 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.194345 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.198699 (+ 4354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:33575: BlockingRecv error: recv got EOF from 127.2.245.193:33575 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:01.570906 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:01.561146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:36475)
0621 06:59:01.561670 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:01.561686 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:01.561843 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:01.562182 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:01.562187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:01.562621 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:01.562789 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.562797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.563844 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.563852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.564776 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.564787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.565571 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.565576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.565764 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.566467 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:01.566485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:01.570698 (+ 4213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
W20260621 06:59:01.571234 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:01.561325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:36475 (local address 127.2.245.254:35783)
0621 06:59:01.561529 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:01.561535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:01.561770 (+ 235us) server_negotiation.cc:407] Connection header received
0621 06:59:01.561968 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:01.561975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:01.562037 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:01.562160 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:01.562930 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.563709 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.564919 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.565369 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.565542 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.570915 (+ 5373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:36475: BlockingRecv error: recv got EOF from 127.2.245.194:36475 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:01 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:01.901643 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:01.892820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:44685)
0621 06:59:01.893328 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:01.893344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:01.893450 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:01.893742 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:01.893745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:01.894150 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:01.894334 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.894342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.895322 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.895326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.896275 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:01.896288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.896915 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.896919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:01.897051 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.897674 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:01.897691 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:01.901273 (+ 3582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":268,"threads_started":1}
W20260621 06:59:01.901726 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:01.892894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:44685 (local address 127.2.245.254:35783)
0621 06:59:01.893065 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:01.893069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:01.893415 (+ 346us) server_negotiation.cc:407] Connection header received
0621 06:59:01.893540 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:01.893544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:01.893602 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:01.893686 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:01.894450 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.895176 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.896409 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:01.896794 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:01.896882 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:01.901451 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:44685: BlockingRecv error: recv got EOF from 127.2.245.195:44685 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:02.208669 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:02.199881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:51645)
0621 06:59:02.200429 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:02.200445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:02.200601 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:02.200980 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:02.200985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:02.201388 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:02.201546 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.201554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.202528 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.202535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.203521 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.203532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.204247 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.204253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.204432 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.204976 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:02.204992 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:02.208427 (+ 3435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":334,"threads_started":1}
W20260621 06:59:02.208844 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:02.199907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:51645 (local address 127.2.245.254:35783)
0621 06:59:02.200155 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:02.200163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:02.200533 (+ 370us) server_negotiation.cc:407] Connection header received
0621 06:59:02.200735 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:02.200741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:02.200845 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:02.200949 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:02.201685 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.202394 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.203671 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.204106 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.204238 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.208613 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:51645: BlockingRecv error: recv got EOF from 127.2.245.193:51645 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:02.581094 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:02.571957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:55825)
0621 06:59:02.572470 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:02.572485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:02.572636 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:02.572982 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:02.572987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:02.573409 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:02.573584 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.573593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.574671 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.574677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.575639 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.575650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.576371 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.576376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.576570 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.577286 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:02.577305 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:02.580788 (+ 3483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
W20260621 06:59:02.581362 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:02.572103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:55825 (local address 127.2.245.254:35783)
0621 06:59:02.572327 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:02.572333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:02.572585 (+ 252us) server_negotiation.cc:407] Connection header received
0621 06:59:02.572777 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:02.572782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:02.572852 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:02.572951 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:02.573732 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.574518 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.575781 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.576226 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.576359 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.581060 (+ 4701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:55825: BlockingRecv error: recv got EOF from 127.2.245.194:55825 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:02 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:02.911478 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:02.902570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:51553)
0621 06:59:02.903164 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:02.903179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:02.903336 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:02.903703 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:02.903707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:02.904149 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:02.904310 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.904319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.905342 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.905348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.906329 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:02.906341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.907222 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.907228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:02.907411 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.907961 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:02.907977 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:02.911171 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":300,"threads_started":1}
W20260621 06:59:02.911612 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:02.902644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:51553 (local address 127.2.245.254:35783)
0621 06:59:02.902892 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:02.902901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:02.903263 (+ 362us) server_negotiation.cc:407] Connection header received
0621 06:59:02.903469 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:02.903472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:02.903529 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:02.903676 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:02.904455 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.905194 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.906486 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:02.907075 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:02.907213 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:02.911352 (+ 4139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:51553: BlockingRecv error: recv got EOF from 127.2.245.195:51553 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:03.218585 12287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:03.209756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:58905)
0621 06:59:03.210306 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:03.210327 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:03.210468 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:03.210846 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:03.210851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:03.211286 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:03.211421 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.211428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.212345 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.212349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.213340 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.213352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.214010 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.214015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.214171 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.214766 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:03.214783 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:03.218234 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":312,"threads_started":1}
W20260621 06:59:03.218676 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:03.209861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:58905 (local address 127.2.245.254:35783)
0621 06:59:03.210121 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:03.210128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:03.210400 (+ 272us) server_negotiation.cc:407] Connection header received
0621 06:59:03.210604 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:03.210609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:03.210689 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:03.210792 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:03.211539 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.212208 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.213468 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.213853 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.213944 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.218450 (+ 4506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:58905: BlockingRecv error: recv got EOF from 127.2.245.193:58905 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:03.591109 12288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:03.582144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:35719)
0621 06:59:03.582678 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:03.582694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:03.582804 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:03.583145 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:03.583148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:03.583498 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:03.583647 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.583655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.584676 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.584680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.585674 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.585687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.586401 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.586406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.586526 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.587221 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:03.587239 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:03.590776 (+ 3537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":314,"threads_started":1}
W20260621 06:59:03.591354 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:03.582318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:35719 (local address 127.2.245.254:35783)
0621 06:59:03.582521 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:03.582526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:03.582766 (+ 240us) server_negotiation.cc:407] Connection header received
0621 06:59:03.582903 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:03.582906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:03.582983 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:03.583064 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:03.583775 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.584538 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.585800 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.586277 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.586370 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.591050 (+ 4680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:35719: BlockingRecv error: recv got EOF from 127.2.245.194:35719 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:03 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.195@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:03.921023 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:03.912462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.195:37677)
0621 06:59:03.912975 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:03.912996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:03.913149 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:03.913510 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:03.913516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:03.913921 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:03.914075 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.914083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.915076 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.915080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.916126 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:03.916138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.916771 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.916775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:03.916890 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.917549 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:03.917566 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:03.920818 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":260,"threads_started":1}
W20260621 06:59:03.921200 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:03.912530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:37677 (local address 127.2.245.254:35783)
0621 06:59:03.912766 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:03.912775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:03.913056 (+ 281us) server_negotiation.cc:407] Connection header received
0621 06:59:03.913267 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:03.913272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:03.913340 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:03.913449 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:03.914196 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.914896 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.916242 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:03.916655 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:03.916776 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:03.920996 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.195:37677: BlockingRecv error: recv got EOF from 127.2.245.195:37677 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 21 06:59:04 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:04 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.193@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:04.229038 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:04.219759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.193:41721)
0621 06:59:04.220230 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:04.220248 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:04.220361 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:04.220710 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:04.220714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:04.221050 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:04.221222 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:04.221230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.222224 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.222228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:04.223253 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:04.223265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.224118 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.224123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:04.224342 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.225136 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:04.225161 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:04.228807 (+ 3646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":225,"threads_started":1}
W20260621 06:59:04.229357 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:04.219910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41721 (local address 127.2.245.254:35783)
0621 06:59:04.220278 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:04.220283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:04.220313 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:59:04.220480 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:04.220483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:04.220544 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:04.220628 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:04.221368 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.222083 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.223386 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.223961 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.224091 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.228995 (+ 4904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.193:41721: BlockingRecv error: recv got EOF from 127.2.245.193:41721 (error 108)
Metrics: {"server-negotiator.queue_time_us":214}
I20260621 06:59:04.586534 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11830
Jun 21 06:59:04 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
Jun 21 06:59:04 dist-test-slave-9zdv krb5kdc[11243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.245.194@KRBTEST.COM for oryx/127.2.245.254@KRBTEST.COM, Server not found in Kerberos database
I20260621 06:59:04.600571 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:04.587729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.2.245.193:37465)
0621 06:59:04.588400 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:04.588407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:04.588434 (+ 27us) server_negotiation.cc:407] Connection header received
0621 06:59:04.588497 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:04.588510 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:04.588742 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:04.588907 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:04.590305 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.591493 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.593139 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.593725 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.593885 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.596659 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:04.596669 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:04.596682 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:04.596725 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:04.599803 (+ 3078us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0621 06:59:04.600208 (+ 405us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44162: Request ticket server kudu/127.2.245.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":341,"threads_started":1}
W20260621 06:59:04.600544 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11830: Not authorized: Client connection negotiation failed: client connection to 127.2.245.193:37465: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260621 06:59:04.600773 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11830
I20260621 06:59:04.602133 12293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:04.592440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35783 (local address 127.2.245.194:50419)
0621 06:59:04.592897 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:04.592912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:04.593080 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:04.593417 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:04.593422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:04.593798 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:04.593952 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:04.593960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.594994 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.595000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:04.595905 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:04.595917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.596626 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.596632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:04.596812 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.597432 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:04.597449 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:04.601719 (+ 4270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.245.254:35783: Server oryx/127.2.245.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":246,"threads_started":1}
W20260621 06:59:04.602167 12248 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:04.592544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:50419 (local address 127.2.245.254:35783)
0621 06:59:04.592756 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:04.592762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:04.593004 (+ 242us) server_negotiation.cc:407] Connection header received
0621 06:59:04.593217 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:04.593222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:04.593288 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:04.593389 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:04.594090 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.594819 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.596068 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.596476 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.596616 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.601927 (+ 5311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.245.194:50419: BlockingRecv error: recv got EOF from 127.2.245.194:50419 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260621 06:59:04.618505 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11981
I20260621 06:59:04.631419 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:04.619738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.2.245.194:42819)
0621 06:59:04.620440 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:04.620449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:04.620483 (+ 34us) server_negotiation.cc:407] Connection header received
0621 06:59:04.620570 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:04.620582 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:04.620910 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:04.621127 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:04.622346 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.623483 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.624784 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.625367 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.625469 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.628039 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:04.628048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:04.628061 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:04.628096 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:04.630657 (+ 2561us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0621 06:59:04.631133 (+ 476us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35424: Request ticket server kudu/127.2.245.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":356,"threads_started":1}
W20260621 06:59:04.631425 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11981: Not authorized: Client connection negotiation failed: client connection to 127.2.245.194:42819: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260621 06:59:04.631522 12294 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35424: Request ticket server kudu/127.2.245.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260621 06:59:04.631567 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11981
I20260621 06:59:04.648247 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12114
I20260621 06:59:04.661279 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:04.649398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.2.245.195:44633)
0621 06:59:04.650023 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:04.650030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:04.650057 (+ 27us) server_negotiation.cc:407] Connection header received
0621 06:59:04.650121 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:04.650128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:04.650394 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:04.650639 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:04.652121 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.653161 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.654436 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:04.655209 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:04.655363 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:04.657913 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:04.657922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:04.657937 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:04.657976 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:04.660473 (+ 2497us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0621 06:59:04.660895 (+ 422us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40874: Request ticket server kudu/127.2.245.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":310,"threads_started":1}
W20260621 06:59:04.661267 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12114: Not authorized: Client connection negotiation failed: client connection to 127.2.245.195:44633: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.245.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260621 06:59:04.661401 12295 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40874: Request ticket server kudu/127.2.245.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260621 06:59:04.661471 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12114
I20260621 06:59:04.678511 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 11761
I20260621 06:59:04.679910 11823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:04.795077 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 11761
2026-06-21T06:59:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21161 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-21T06:59:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:59:04Z Disabled control of system clock
I20260621 06:59:04.871189 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:42967
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260621 06:59:05.196614 12300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:05.196908 12300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:05.197049 12300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:05.204380 12300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:59:05.204483 12300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:05.204569 12300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:05.204682 12300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:59:05.204744 12300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:59:05.213822 12300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:42967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:05.215126 12300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:05.216859 12300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:05.229101 12305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:05.229071 12306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:05.229072 12308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:05.229653 12300 server_base.cc:1034] running on GCE node
I20260621 06:59:05.230463 12300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:05.231884 12300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:05.233074 12300 hybrid_clock.cc:648] HybridClock initialized: now 1782025145233040 us; error 37 us; skew 500 ppm
I20260621 06:59:05.233469 12300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:05.235708 12300 webserver.cc:460] Webserver started at http://127.2.245.254:37577/ using document root <none> and password file <none>
I20260621 06:59:05.236421 12300 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:05.236531 12300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:05.236855 12300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:05.239785 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "8fbc77b2943b749b18c7da28e995e927"
server_key_iv: "8cb0fba06eefebbec629e3359ed49005"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.240558 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "8fbc77b2943b749b18c7da28e995e927"
server_key_iv: "8cb0fba06eefebbec629e3359ed49005"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.245646 12300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260621 06:59:05.249081 12314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:05.250550 12300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 06:59:05.250741 12300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "8fbc77b2943b749b18c7da28e995e927"
server_key_iv: "8cb0fba06eefebbec629e3359ed49005"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.251011 12300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:05.262889 12300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:05.263736 12300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:05.264024 12300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:05.282214 12300 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:42967
I20260621 06:59:05.282222 12365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:42967 every 8 connection(s)
I20260621 06:59:05.283625 12300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:59:05.287416 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:05.291790 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12300
I20260621 06:59:05.291994 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:59:05.292659 3031 external_mini_cluster.cc:1428] Setting key a5965d98be115eb132edf002c3bfc30d
I20260621 06:59:05.299476 12366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc: Bootstrap starting.
I20260621 06:59:05.303380 12366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:05.304572 12366 log.cc:824] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:05.307786 12366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc: No bootstrap required, opened a new log
I20260621 06:59:05.308077 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:05.294726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.2.245.254:42967)
0621 06:59:05.295895 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:05.295914 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:05.295975 (+ 61us) server_negotiation.cc:407] Connection header received
0621 06:59:05.298363 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:05.298393 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:05.299016 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:05.299463 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:05.300029 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.302049 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.303481 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.304182 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.304409 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:05.305611 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:05.305623 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:05.305650 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:05.306306 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:05.306393 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:05.306413 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:05.306499 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:05.306755 (+ 256us) server_negotiation.cc:299] Negotiation successful
0621 06:59:05.307115 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":392,"threads_started":1}
I20260621 06:59:05.315613 12366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } }
I20260621 06:59:05.316100 12366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:05.316238 12366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7371f8d9d9fb4ad3b131c1a2cbe982fc, State: Initialized, Role: FOLLOWER
I20260621 06:59:05.316900 12366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } }
I20260621 06:59:05.317147 12366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:59:05.317286 12366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:59:05.317457 12366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:05.319667 12366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } }
I20260621 06:59:05.320261 12366 leader_election.cc:302] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7371f8d9d9fb4ad3b131c1a2cbe982fc; no voters:
I20260621 06:59:05.320879 12366 leader_election.cc:288] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:59:05.321023 12371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:59:05.321735 12371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [term 1 LEADER]: Becoming Leader. State: Replica: 7371f8d9d9fb4ad3b131c1a2cbe982fc, State: Running, Role: LEADER
I20260621 06:59:05.322257 12371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } }
I20260621 06:59:05.323004 12366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:59:05.325093 12373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7371f8d9d9fb4ad3b131c1a2cbe982fc. Latest consensus state: current_term: 1 leader_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } } }
I20260621 06:59:05.325278 12372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371f8d9d9fb4ad3b131c1a2cbe982fc" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42967 } } }
I20260621 06:59:05.325609 12373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:05.325601 12372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:05.328204 12378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:59:05.332643 12378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:59:05.340916 12378 catalog_manager.cc:1347] Generated new cluster ID: 77da03bdc1e94e4ba50753a00023c566
I20260621 06:59:05.341055 12378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:59:05.350281 12378 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:59:05.352177 12378 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:59:05.361416 12378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7371f8d9d9fb4ad3b131c1a2cbe982fc: Generated new TSK 0
I20260621 06:59:05.362476 12378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:59:05.371093 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--builtin_ntp_servers=127.2.245.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260621 06:59:05.701993 12390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:05.702280 12390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:05.702361 12390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:05.709687 12390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:05.709785 12390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:05.709887 12390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:59:05.719103 12390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:05.720381 12390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:05.722131 12390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:05.735589 12396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:05.735633 12398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:05.735666 12395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:05.736822 12390 server_base.cc:1034] running on GCE node
I20260621 06:59:05.737411 12390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:05.738303 12390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:05.739504 12390 hybrid_clock.cc:648] HybridClock initialized: now 1782025145739458 us; error 37 us; skew 500 ppm
I20260621 06:59:05.739830 12390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:05.746191 12390 webserver.cc:460] Webserver started at http://127.2.245.193:46103/ using document root <none> and password file <none>
I20260621 06:59:05.746883 12390 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:05.747051 12390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:05.747370 12390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:05.750090 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "d9b825af69fa4c2bb52f4e9f227ae770"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "6fb6f309920cf7edae4f3a92b180e29a"
server_key_iv: "a4ff882ddab27693c3e8a88b5e7b2adb"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.750825 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "d9b825af69fa4c2bb52f4e9f227ae770"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "6fb6f309920cf7edae4f3a92b180e29a"
server_key_iv: "a4ff882ddab27693c3e8a88b5e7b2adb"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.756072 12390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260621 06:59:05.759473 12404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:05.760787 12390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:59:05.760962 12390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "d9b825af69fa4c2bb52f4e9f227ae770"
format_stamp: "Formatted at 2026-06-21 06:59:05 on dist-test-slave-9zdv"
server_key: "6fb6f309920cf7edae4f3a92b180e29a"
server_key_iv: "a4ff882ddab27693c3e8a88b5e7b2adb"
server_key_version: "encryptionkey@0"
I20260621 06:59:05.761178 12390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:05.779211 12390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:05.780040 12390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:05.780333 12390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:05.781284 12390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:05.782769 12390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:05.782894 12390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:05.783033 12390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:05.783118 12390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:05.815402 12390 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:40341
I20260621 06:59:05.815454 12516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:40341 every 8 connection(s)
I20260621 06:59:05.816726 12390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:59:05.823822 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12390
I20260621 06:59:05.824013 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:59:05.824733 3031 external_mini_cluster.cc:1428] Setting key 459cd923b826ddc7846510b89baac8b0
I20260621 06:59:05.827497 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--builtin_ntp_servers=127.2.245.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260621 06:59:05.829784 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:05.819417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:55679 (local address 127.2.245.254:42967)
0621 06:59:05.819861 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:05.819867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:05.821017 (+ 1150us) server_negotiation.cc:407] Connection header received
0621 06:59:05.822752 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:05.822757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:05.822831 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:05.822931 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:05.824078 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.824931 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.826526 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.827093 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.827229 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:05.828807 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:05.828816 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:05.828824 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:05.828849 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:05.828903 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:05.828921 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:05.829012 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:05.829227 (+ 215us) server_negotiation.cc:299] Negotiation successful
0621 06:59:05.829347 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":195,"threads_started":1}
I20260621 06:59:05.830597 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:05.819997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42967 (local address 127.2.245.193:55679)
0621 06:59:05.820842 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:05.820886 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:05.822464 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:05.823188 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:05.823196 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:05.823314 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:05.823861 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:05.823922 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.825087 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.825092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:05.826365 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:05.826385 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:05.827332 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:05.827339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:05.827593 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:05.828463 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:05.828465 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:05.828503 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:05.828577 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:05.829028 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:05.829035 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:05.829042 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 06:59:05.829185 (+ 143us) client_negotiation.cc:240] Negotiation successful
0621 06:59:05.829497 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":258,"threads_started":1}
I20260621 06:59:05.832142 12517 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42967
I20260621 06:59:05.832422 12517 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:05.833173 12517 heartbeater.cc:511] Master 127.2.245.254:42967 requested a full tablet report, sending...
I20260621 06:59:05.835269 12331 ts_manager.cc:194] Registered new tserver with Master: d9b825af69fa4c2bb52f4e9f227ae770 (127.2.245.193:40341)
I20260621 06:59:05.836781 12331 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.193:55679
W20260621 06:59:06.148976 12521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:06.149295 12521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:06.149458 12521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:06.156781 12521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:06.156909 12521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:06.157074 12521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:59:06.166122 12521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:06.167407 12521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:06.169164 12521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:06.181936 12529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:06.181999 12527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:06.182022 12526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:06.182662 12521 server_base.cc:1034] running on GCE node
I20260621 06:59:06.183276 12521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:06.184109 12521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:06.185302 12521 hybrid_clock.cc:648] HybridClock initialized: now 1782025146185262 us; error 45 us; skew 500 ppm
I20260621 06:59:06.185626 12521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:06.187775 12521 webserver.cc:460] Webserver started at http://127.2.245.194:32929/ using document root <none> and password file <none>
I20260621 06:59:06.188445 12521 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:06.188556 12521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:06.188874 12521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:06.191684 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "a4f34453fbc74b8888cbcf16f580d6b8"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "6cf9b23690edbbc6f90acf68e3f01c74"
server_key_iv: "d6e8d2cec2ad2af9245f117729c07f72"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.192451 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "a4f34453fbc74b8888cbcf16f580d6b8"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "6cf9b23690edbbc6f90acf68e3f01c74"
server_key_iv: "d6e8d2cec2ad2af9245f117729c07f72"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.197491 12521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 06:59:06.200851 12535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:06.202117 12521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 06:59:06.202292 12521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "a4f34453fbc74b8888cbcf16f580d6b8"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "6cf9b23690edbbc6f90acf68e3f01c74"
server_key_iv: "d6e8d2cec2ad2af9245f117729c07f72"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.202512 12521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:06.219609 12521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:06.220467 12521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:06.220706 12521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:06.221529 12521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:06.222899 12521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:06.223047 12521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:06.223138 12521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:06.223217 12521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:06.256050 12521 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:43929
I20260621 06:59:06.256083 12647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:43929 every 8 connection(s)
I20260621 06:59:06.257303 12521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:59:06.260699 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12521
I20260621 06:59:06.260897 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:59:06.261415 3031 external_mini_cluster.cc:1428] Setting key 46d3981cbac791ecd320e542c9da365e
I20260621 06:59:06.264086 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--builtin_ntp_servers=127.2.245.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260621 06:59:06.270257 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.259827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:39027 (local address 127.2.245.254:42967)
0621 06:59:06.259998 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.260002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.261646 (+ 1644us) server_negotiation.cc:407] Connection header received
0621 06:59:06.263638 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.263642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.263727 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.263833 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.265041 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.265876 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.267485 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.267890 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.268023 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.269436 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:06.269441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:06.269444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:06.269463 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:06.269493 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:06.269504 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:06.269603 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:06.269833 (+ 230us) server_negotiation.cc:299] Negotiation successful
0621 06:59:06.269943 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:59:06.271420 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.260434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42967 (local address 127.2.245.194:39027)
0621 06:59:06.261452 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:06.261524 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:06.263389 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:06.264055 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:06.264063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:06.264208 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:06.264881 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:06.264897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.266029 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.266034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:06.267334 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:06.267350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.268143 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.268150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:06.268381 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.269116 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:06.269119 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:06.269158 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:06.269239 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:06.269601 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:06.269614 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:06.269624 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 06:59:06.269826 (+ 202us) client_negotiation.cc:240] Negotiation successful
0621 06:59:06.270236 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":337,"threads_started":1}
I20260621 06:59:06.272935 12648 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42967
I20260621 06:59:06.273243 12648 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:06.274012 12648 heartbeater.cc:511] Master 127.2.245.254:42967 requested a full tablet report, sending...
I20260621 06:59:06.275455 12331 ts_manager.cc:194] Registered new tserver with Master: a4f34453fbc74b8888cbcf16f580d6b8 (127.2.245.194:43929)
I20260621 06:59:06.276367 12331 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.194:39027
W20260621 06:59:06.591444 12651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:06.591812 12651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:06.591958 12651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:06.599416 12651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:06.599547 12651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:06.599722 12651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:59:06.609079 12651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:06.610384 12651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:06.612196 12651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:06.625376 12656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:06.625408 12657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:06.625627 12659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:06.626389 12651 server_base.cc:1034] running on GCE node
I20260621 06:59:06.627104 12651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:06.628096 12651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:06.629289 12651 hybrid_clock.cc:648] HybridClock initialized: now 1782025146629232 us; error 57 us; skew 500 ppm
I20260621 06:59:06.629595 12651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:06.631765 12651 webserver.cc:460] Webserver started at http://127.2.245.195:41545/ using document root <none> and password file <none>
I20260621 06:59:06.632454 12651 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:06.632575 12651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:06.632902 12651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:06.635792 12651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "a1eae7473bce4210b9f4a1dc12c8078e"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "c611b1c75bcc8106be880f5d0a4d2372"
server_key_iv: "2401370d76d1a1bc99461bdecd3bb5e9"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.636577 12651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "a1eae7473bce4210b9f4a1dc12c8078e"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "c611b1c75bcc8106be880f5d0a4d2372"
server_key_iv: "2401370d76d1a1bc99461bdecd3bb5e9"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.641851 12651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 06:59:06.645255 12665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:06.646528 12651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 06:59:06.646698 12651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "a1eae7473bce4210b9f4a1dc12c8078e"
format_stamp: "Formatted at 2026-06-21 06:59:06 on dist-test-slave-9zdv"
server_key: "c611b1c75bcc8106be880f5d0a4d2372"
server_key_iv: "2401370d76d1a1bc99461bdecd3bb5e9"
server_key_version: "encryptionkey@0"
I20260621 06:59:06.646925 12651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:06.663203 12651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:06.663988 12651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:06.664292 12651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:06.665272 12651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:06.666664 12651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:06.666792 12651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:06.666916 12651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:06.667061 12651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260621 06:59:06.699532 12651 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:39357
I20260621 06:59:06.699554 12777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:39357 every 8 connection(s)
I20260621 06:59:06.700798 12651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:59:06.708025 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12651
I20260621 06:59:06.708205 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:59:06.708788 3031 external_mini_cluster.cc:1428] Setting key ec3b9bed71e6ab2c94a2257720670958
I20260621 06:59:06.714010 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.703420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:35841 (local address 127.2.245.254:42967)
0621 06:59:06.703605 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.703609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.705024 (+ 1415us) server_negotiation.cc:407] Connection header received
0621 06:59:06.707089 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.707094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.707186 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.707308 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.708707 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.709497 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.711651 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.712085 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.712183 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.713377 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:06.713381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:06.713384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:06.713401 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:06.713431 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:06.713442 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:06.713510 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:06.713701 (+ 191us) server_negotiation.cc:299] Negotiation successful
0621 06:59:06.713784 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 06:59:06.715097 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.703949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42967 (local address 127.2.245.195:35841)
0621 06:59:06.704863 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:06.704908 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:06.706704 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:06.707540 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:06.707548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:06.707670 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:06.708526 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:06.708552 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.709666 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.709672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:06.711503 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:06.711527 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.712277 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.712282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:06.712471 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.713098 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:06.713100 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:06.713138 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:06.713213 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:06.713525 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:06.713532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:06.713538 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 06:59:06.713674 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 06:59:06.713964 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":289,"threads_started":1}
I20260621 06:59:06.716401 12778 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42967
I20260621 06:59:06.716681 12778 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:06.717458 12778 heartbeater.cc:511] Master 127.2.245.254:42967 requested a full tablet report, sending...
I20260621 06:59:06.718819 12331 ts_manager.cc:194] Registered new tserver with Master: a1eae7473bce4210b9f4a1dc12c8078e (127.2.245.195:39357)
I20260621 06:59:06.719621 12331 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.195:35841
I20260621 06:59:06.723940 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 06:59:06.729976 12519 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:06.728408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49596 (local address 127.2.245.254:42967)
0621 06:59:06.728671 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.728680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.729014 (+ 334us) server_negotiation.cc:407] Connection header received
0621 06:59:06.729151 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.729155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.729240 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.729323 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.729753 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49596: BlockingRecv error: recv got EOF from 127.0.0.1:49596 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 06:59:06.730748 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12390
I20260621 06:59:06.738000 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.731679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.2.245.193:40341)
0621 06:59:06.732174 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.732187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.732220 (+ 33us) server_negotiation.cc:407] Connection header received
0621 06:59:06.732273 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.732280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.732480 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.732601 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.732890 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.733845 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.735070 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.735618 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.735731 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.736640 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:06.736647 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:06.736660 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:06.737253 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:06.737302 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:06.737320 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:06.737407 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:06.737501 (+ 94us) server_negotiation.cc:299] Negotiation successful
0621 06:59:06.737641 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":273,"threads_started":1}
I20260621 06:59:06.739270 12512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:06.822070 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12390
I20260621 06:59:06.838534 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12521
I20260621 06:59:06.846603 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.839559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.2.245.194:43929)
0621 06:59:06.840135 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.840142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.840168 (+ 26us) server_negotiation.cc:407] Connection header received
0621 06:59:06.840229 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.840236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.840462 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.840656 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.841071 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.842131 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.843408 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.843997 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.844140 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.845146 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:06.845154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:06.845167 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:06.845783 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:06.845830 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:06.845849 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:06.845978 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:06.846124 (+ 146us) server_negotiation.cc:299] Negotiation successful
0621 06:59:06.846268 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":306,"threads_started":1}
I20260621 06:59:06.847894 12643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:06.928464 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12521
I20260621 06:59:06.944903 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12651
I20260621 06:59:06.952920 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:06.945823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.2.245.195:39357)
0621 06:59:06.946460 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:06.946467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:06.946497 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:59:06.946554 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:06.946560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:06.946789 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:06.946915 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:06.947374 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.948469 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.949655 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:06.950234 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:06.950331 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:06.951467 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:06.951475 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:06.951487 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:06.952153 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:06.952198 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:06.952216 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:06.952302 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:06.952401 (+ 99us) server_negotiation.cc:299] Negotiation successful
0621 06:59:06.952541 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
I20260621 06:59:06.954244 12773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:07.039661 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12651
I20260621 06:59:07.055958 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12300
I20260621 06:59:07.057124 12361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:07.161599 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12300
2026-06-21T06:59:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2361 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-21T06:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:59:07Z Disabled control of system clock
I20260621 06:59:07.233398 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38185
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260621 06:59:07.553544 12797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:07.553830 12797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:07.553992 12797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:07.561126 12797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:59:07.561259 12797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:07.561353 12797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:07.561455 12797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:59:07.561537 12797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:59:07.570601 12797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:38185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:07.571920 12797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:07.573814 12797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:07.585780 12805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:07.585862 12802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:07.585851 12803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:07.591813 12797 server_base.cc:1034] running on GCE node
I20260621 06:59:07.592651 12797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:07.594110 12797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:07.595407 12797 hybrid_clock.cc:648] HybridClock initialized: now 1782025147595376 us; error 48 us; skew 500 ppm
I20260621 06:59:07.595793 12797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:07.598053 12797 webserver.cc:460] Webserver started at http://127.2.245.254:43957/ using document root <none> and password file <none>
I20260621 06:59:07.598728 12797 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:07.598855 12797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:07.599223 12797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:07.601941 12797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "f9e0698c3a6c4df4af2f11a9d45342f0"
format_stamp: "Formatted at 2026-06-21 06:59:07 on dist-test-slave-9zdv"
server_key: "eb7483052b05c6c4974a974849e0075b"
server_key_iv: "3ed96356c20b3e7153fefbac676b938d"
server_key_version: "encryptionkey@0"
I20260621 06:59:07.602699 12797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "f9e0698c3a6c4df4af2f11a9d45342f0"
format_stamp: "Formatted at 2026-06-21 06:59:07 on dist-test-slave-9zdv"
server_key: "eb7483052b05c6c4974a974849e0075b"
server_key_iv: "3ed96356c20b3e7153fefbac676b938d"
server_key_version: "encryptionkey@0"
I20260621 06:59:07.607803 12797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 06:59:07.611099 12811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:07.612543 12797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 06:59:07.612759 12797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "f9e0698c3a6c4df4af2f11a9d45342f0"
format_stamp: "Formatted at 2026-06-21 06:59:07 on dist-test-slave-9zdv"
server_key: "eb7483052b05c6c4974a974849e0075b"
server_key_iv: "3ed96356c20b3e7153fefbac676b938d"
server_key_version: "encryptionkey@0"
I20260621 06:59:07.612977 12797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:07.618899 12797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:07.619715 12797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:07.620044 12797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:07.638468 12797 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:38185
I20260621 06:59:07.638510 12862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:38185 every 8 connection(s)
I20260621 06:59:07.639842 12797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:59:07.643476 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:07.644352 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12797
I20260621 06:59:07.644568 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:59:07.645241 3031 external_mini_cluster.cc:1428] Setting key c15ea92f012feceebd60bd6263ca2d71
I20260621 06:59:07.654279 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:07.647322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.2.245.254:38185)
0621 06:59:07.648543 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:07.648559 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:07.648613 (+ 54us) server_negotiation.cc:407] Connection header received
0621 06:59:07.650198 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:07.650234 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:07.650797 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:07.651370 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:07.652377 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:07.652385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:07.652401 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:07.652417 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:07.652490 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:07.652511 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:07.652721 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:07.652854 (+ 133us) server_negotiation.cc:299] Negotiation successful
0621 06:59:07.653121 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":416,"threads_started":1}
I20260621 06:59:07.658098 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0: Bootstrap starting.
I20260621 06:59:07.660976 12863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:07.661919 12863 log.cc:824] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:07.664692 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0: No bootstrap required, opened a new log
I20260621 06:59:07.671885 12863 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } }
I20260621 06:59:07.672219 12863 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:07.672323 12863 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e0698c3a6c4df4af2f11a9d45342f0, State: Initialized, Role: FOLLOWER
I20260621 06:59:07.672885 12863 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } }
I20260621 06:59:07.673105 12863 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:59:07.673264 12863 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:59:07.673430 12863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:07.675716 12863 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } }
I20260621 06:59:07.676294 12863 leader_election.cc:302] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9e0698c3a6c4df4af2f11a9d45342f0; no voters:
I20260621 06:59:07.676867 12863 leader_election.cc:288] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:59:07.676992 12868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:59:07.677739 12868 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [term 1 LEADER]: Becoming Leader. State: Replica: f9e0698c3a6c4df4af2f11a9d45342f0, State: Running, Role: LEADER
I20260621 06:59:07.678335 12868 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } }
I20260621 06:59:07.678921 12863 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:59:07.681067 12869 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } } }
I20260621 06:59:07.681463 12869 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:07.683210 12870 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9e0698c3a6c4df4af2f11a9d45342f0. Latest consensus state: current_term: 1 leader_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e0698c3a6c4df4af2f11a9d45342f0" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 38185 } } }
I20260621 06:59:07.683605 12870 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0 [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:07.684242 12876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:59:07.690275 12876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:59:07.701838 12876 catalog_manager.cc:1347] Generated new cluster ID: 4afc0902b11849d4a28a9d7d46dbd0a9
I20260621 06:59:07.701947 12876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:59:07.723524 12876 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:59:07.724013 12876 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:59:07.731904 12876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9e0698c3a6c4df4af2f11a9d45342f0: Generated new TSK 0
I20260621 06:59:07.732748 12876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 06:59:07.740670 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--builtin_ntp_servers=127.2.245.212:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260621 06:59:08.080593 12887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:08.080874 12887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:08.081056 12887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:08.087957 12887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:08.088070 12887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:08.088234 12887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:59:08.096909 12887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:08.098166 12887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:08.099903 12887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:08.113054 12892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.112924 12893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.112982 12895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:08.114126 12887 server_base.cc:1034] running on GCE node
I20260621 06:59:08.114775 12887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:08.115726 12887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:08.116947 12887 hybrid_clock.cc:648] HybridClock initialized: now 1782025148116883 us; error 58 us; skew 500 ppm
I20260621 06:59:08.117278 12887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:08.119568 12887 webserver.cc:460] Webserver started at http://127.2.245.193:36461/ using document root <none> and password file <none>
I20260621 06:59:08.120232 12887 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:08.120357 12887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:08.120677 12887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:08.124826 12887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "c02ea1aeef6a4cd38296524109d1dc15"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "aef8b4769cca34840f22096eaf4511ed"
server_key_iv: "7d5af774c544901290d6b85af25324a8"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.125613 12887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "c02ea1aeef6a4cd38296524109d1dc15"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "aef8b4769cca34840f22096eaf4511ed"
server_key_iv: "7d5af774c544901290d6b85af25324a8"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.130635 12887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 06:59:08.133893 12901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.135250 12887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:59:08.135433 12887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "c02ea1aeef6a4cd38296524109d1dc15"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "aef8b4769cca34840f22096eaf4511ed"
server_key_iv: "7d5af774c544901290d6b85af25324a8"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.135651 12887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:08.140996 12887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:08.141744 12887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:08.142005 12887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:08.142817 12887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:08.144212 12887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:08.144340 12887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.144470 12887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:08.144555 12887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.177076 12887 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:41419
I20260621 06:59:08.177137 13013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:41419 every 8 connection(s)
I20260621 06:59:08.178443 12887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:59:08.185804 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 12887
I20260621 06:59:08.186096 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:59:08.186704 3031 external_mini_cluster.cc:1428] Setting key 84d29e5cb6e01eae25082344856f3bc7
I20260621 06:59:08.187314 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:08.181131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:56181 (local address 127.2.245.254:38185)
0621 06:59:08.181549 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:08.181554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:08.182782 (+ 1228us) server_negotiation.cc:407] Connection header received
0621 06:59:08.184796 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:08.184802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:08.184878 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:08.185000 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:08.186490 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:08.186495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:08.186500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:08.186524 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:08.186564 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:08.186576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:08.186701 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:08.186879 (+ 178us) server_negotiation.cc:299] Negotiation successful
0621 06:59:08.186968 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":200,"threads_started":1}
I20260621 06:59:08.188184 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:08.181719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38185 (local address 127.2.245.193:56181)
0621 06:59:08.182600 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:08.182641 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:08.184505 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:08.185236 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:08.185244 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:08.185374 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:08.186184 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:08.186188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:08.186245 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:08.186328 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:08.186674 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:08.186683 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:08.186691 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 06:59:08.186979 (+ 288us) client_negotiation.cc:240] Negotiation successful
0621 06:59:08.187250 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":270,"threads_started":1}
I20260621 06:59:08.189638 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--builtin_ntp_servers=127.2.245.212:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260621 06:59:08.189680 13014 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38185
I20260621 06:59:08.190006 13014 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:08.190557 13014 heartbeater.cc:511] Master 127.2.245.254:38185 requested a full tablet report, sending...
I20260621 06:59:08.192510 12828 ts_manager.cc:194] Registered new tserver with Master: c02ea1aeef6a4cd38296524109d1dc15 (127.2.245.193:41419)
W20260621 06:59:08.506186 13018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:08.506505 13018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:08.506672 13018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:08.513873 13018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:08.514011 13018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:08.514190 13018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:59:08.523360 13018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:08.524672 13018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:08.526372 13018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:08.539109 13026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.539072 13024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.539242 13023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:08.540017 13018 server_base.cc:1034] running on GCE node
I20260621 06:59:08.540629 13018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:08.541471 13018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:08.542663 13018 hybrid_clock.cc:648] HybridClock initialized: now 1782025148542630 us; error 30 us; skew 500 ppm
I20260621 06:59:08.543064 13018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:08.545271 13018 webserver.cc:460] Webserver started at http://127.2.245.194:33877/ using document root <none> and password file <none>
I20260621 06:59:08.545943 13018 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:08.546068 13018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:08.546425 13018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:08.549289 13018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "aed2982ea6904c1d9db068911390d6f2"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0f1f8ef630cdcac15f61f137b234a1b8"
server_key_iv: "678e3b58358efad953675beb54ede02b"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.550058 13018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "aed2982ea6904c1d9db068911390d6f2"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0f1f8ef630cdcac15f61f137b234a1b8"
server_key_iv: "678e3b58358efad953675beb54ede02b"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.554885 13018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260621 06:59:08.558159 13032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.559506 13018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:59:08.559675 13018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "aed2982ea6904c1d9db068911390d6f2"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0f1f8ef630cdcac15f61f137b234a1b8"
server_key_iv: "678e3b58358efad953675beb54ede02b"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.559876 13018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:08.565168 13018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:08.565930 13018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:08.566198 13018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:08.567153 13018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:08.568472 13018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:08.568567 13018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.568711 13018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:08.568790 13018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.601176 13018 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:41403
I20260621 06:59:08.601222 13144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:41403 every 8 connection(s)
I20260621 06:59:08.602483 13018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:59:08.610811 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:08.605161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:34269 (local address 127.2.245.254:38185)
0621 06:59:08.605348 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:08.605353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:08.606792 (+ 1439us) server_negotiation.cc:407] Connection header received
0621 06:59:08.608558 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:08.608563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:08.608655 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:08.608774 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:08.610048 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:08.610054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:08.610058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:08.610077 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:08.610115 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:08.610126 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:08.610251 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:08.610449 (+ 198us) server_negotiation.cc:299] Negotiation successful
0621 06:59:08.610518 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260621 06:59:08.611053 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13018
I20260621 06:59:08.611286 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:59:08.611757 3031 external_mini_cluster.cc:1428] Setting key 2535a4dc1ae7e0eb754bdb1d981e8b92
I20260621 06:59:08.611901 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:08.605698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38185 (local address 127.2.245.194:34269)
0621 06:59:08.606613 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:08.606650 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:08.608289 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:08.608996 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:08.609005 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:08.609094 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:08.609756 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:08.609759 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:08.609800 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:08.609881 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:08.610214 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:08.610223 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:08.610231 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 06:59:08.610541 (+ 310us) client_negotiation.cc:240] Negotiation successful
0621 06:59:08.610797 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":266,"threads_started":1}
I20260621 06:59:08.613284 13145 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38185
I20260621 06:59:08.613613 13145 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:08.614140 13145 heartbeater.cc:511] Master 127.2.245.254:38185 requested a full tablet report, sending...
I20260621 06:59:08.614570 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--builtin_ntp_servers=127.2.245.212:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260621 06:59:08.615527 12827 ts_manager.cc:194] Registered new tserver with Master: aed2982ea6904c1d9db068911390d6f2 (127.2.245.194:41403)
W20260621 06:59:08.936528 13148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:08.936878 13148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:08.936973 13148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:08.944391 13148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:08.944515 13148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:08.944623 13148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:59:08.953892 13148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:08.955190 13148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:08.956965 13148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:08.969736 13153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.969830 13154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:08.969853 13156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:08.970494 13148 server_base.cc:1034] running on GCE node
I20260621 06:59:08.971181 13148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:08.972126 13148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:08.973320 13148 hybrid_clock.cc:648] HybridClock initialized: now 1782025148973261 us; error 53 us; skew 500 ppm
I20260621 06:59:08.973627 13148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 06:59:08.976058 13148 webserver.cc:460] Webserver started at http://127.2.245.195:40145/ using document root <none> and password file <none>
I20260621 06:59:08.976734 13148 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:08.976856 13148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:08.977185 13148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:08.980475 13148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "9eff72ad18cb428fa9ec8178180cdabe"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0424d7c906b8633434a1541386fe75b3"
server_key_iv: "6f75fb9f11cc1bbccbcd6bc0ec10e155"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.981530 13148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "9eff72ad18cb428fa9ec8178180cdabe"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0424d7c906b8633434a1541386fe75b3"
server_key_iv: "6f75fb9f11cc1bbccbcd6bc0ec10e155"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.987346 13148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260621 06:59:08.991165 13162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:08.992494 13148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:59:08.992631 13148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "9eff72ad18cb428fa9ec8178180cdabe"
format_stamp: "Formatted at 2026-06-21 06:59:08 on dist-test-slave-9zdv"
server_key: "0424d7c906b8633434a1541386fe75b3"
server_key_iv: "6f75fb9f11cc1bbccbcd6bc0ec10e155"
server_key_version: "encryptionkey@0"
I20260621 06:59:08.992828 13148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:08.998672 13148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:08.999558 13148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:08.999804 13148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:09.000701 13148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:09.002130 13148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:09.002214 13148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:09.002321 13148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:09.002375 13148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:09.035655 13148 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:38753
I20260621 06:59:09.035709 13274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:38753 every 8 connection(s)
I20260621 06:59:09.036985 13148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:59:09.037375 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13148
I20260621 06:59:09.037570 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:59:09.038192 3031 external_mini_cluster.cc:1428] Setting key 2e0efde32c92491e1e8b7e39acd45f99
I20260621 06:59:09.045599 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:09.039908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:49701 (local address 127.2.245.254:38185)
0621 06:59:09.040085 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:09.040093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:09.042000 (+ 1907us) server_negotiation.cc:407] Connection header received
0621 06:59:09.043685 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:09.043689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:09.043741 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:09.043827 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:09.045018 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:09.045023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:09.045026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:09.045041 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:09.045072 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:09.045083 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:09.045181 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:09.045345 (+ 164us) server_negotiation.cc:299] Negotiation successful
0621 06:59:09.045393 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 06:59:09.046489 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:09.040715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:38185 (local address 127.2.245.195:49701)
0621 06:59:09.041863 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:09.041899 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:09.043449 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:09.044021 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:09.044029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:09.044119 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:09.044749 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 06:59:09.044752 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:09.044786 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 06:59:09.044868 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:09.045167 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:09.045173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:09.045179 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 06:59:09.045391 (+ 212us) client_negotiation.cc:240] Negotiation successful
0621 06:59:09.045577 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":331,"threads_started":1}
I20260621 06:59:09.047730 13275 heartbeater.cc:347] Connected to a master server at 127.2.245.254:38185
I20260621 06:59:09.048004 13275 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:09.048537 13275 heartbeater.cc:511] Master 127.2.245.254:38185 requested a full tablet report, sending...
I20260621 06:59:09.049799 12827 ts_manager.cc:194] Registered new tserver with Master: 9eff72ad18cb428fa9ec8178180cdabe (127.2.245.195:38753)
I20260621 06:59:09.053678 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 06:59:09.059706 13016 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 06:59:09.058251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.2.245.254:38185)
0621 06:59:09.058405 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:09.058409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:09.058993 (+ 584us) server_negotiation.cc:407] Connection header received
0621 06:59:09.059111 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:09.059115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:09.059166 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:09.059244 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:09.059537 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49104: BlockingRecv error: recv got EOF from 127.0.0.1:49104 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 06:59:09.060492 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12887
I20260621 06:59:09.063854 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:09.061456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.2.245.193:41419)
0621 06:59:09.061947 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:09.061962 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:09.062000 (+ 38us) server_negotiation.cc:407] Connection header received
0621 06:59:09.062055 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:09.062061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:09.062250 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:09.062373 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:09.063242 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:09.063247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:09.063265 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:09.063280 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:09.063320 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:09.063333 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:09.063433 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:09.063565 (+ 132us) server_negotiation.cc:299] Negotiation successful
0621 06:59:09.063639 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":244,"threads_started":1}
I20260621 06:59:09.065014 13009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:09.145215 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12887
I20260621 06:59:09.161379 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13018
I20260621 06:59:09.165115 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:09.162254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.2.245.194:41403)
0621 06:59:09.162706 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:09.162712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:09.162736 (+ 24us) server_negotiation.cc:407] Connection header received
0621 06:59:09.162799 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:09.162805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:09.163102 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:09.163278 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:09.164373 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:09.164380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:09.164391 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:09.164408 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:09.164445 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:09.164458 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:09.164624 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:09.164752 (+ 128us) server_negotiation.cc:299] Negotiation successful
0621 06:59:09.164853 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":210,"threads_started":1}
I20260621 06:59:09.166285 13140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:09.265143 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13018
I20260621 06:59:09.280973 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13148
I20260621 06:59:09.284906 13288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:09.282028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.2.245.195:38753)
0621 06:59:09.282703 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:09.282710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:09.282740 (+ 30us) server_negotiation.cc:407] Connection header received
0621 06:59:09.282824 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:09.282831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:09.283108 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:09.283241 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:09.284264 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:09.284271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:09.284282 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 06:59:09.284293 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:09.284331 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 06:59:09.284344 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:09.284472 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:09.284521 (+ 49us) server_negotiation.cc:299] Negotiation successful
0621 06:59:09.284596 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":347,"threads_started":1}
I20260621 06:59:09.286176 13270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:09.365957 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13148
I20260621 06:59:09.381652 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 12797
I20260621 06:59:09.382906 12858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:09.484355 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 12797
2026-06-21T06:59:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2322 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:59:09 dist-test-slave-9zdv krb5kdc[13293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:59:09 dist-test-slave-9zdv krb5kdc[13293](info): set up 2 sockets
Jun 21 06:59:09 dist-test-slave-9zdv krb5kdc[13293](info): commencing operation
krb5kdc: starting...
W20260621 06:59:10.604106 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:59:10 dist-test-slave-9zdv krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:59:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:59:10.832034 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34423
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:34423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:59:11.162884 13306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:11.163349 13306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:11.163442 13306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:11.170707 13306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:59:11.170821 13306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:11.170868 13306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:11.170917 13306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:59:11.170980 13306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:59:11.180737 13306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:34423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:34423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:11.181962 13306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:11.183285 13306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:11.195148 13314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:11.195195 13312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:11.195111 13311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:11.200462 13306 server_base.cc:1034] running on GCE node
I20260621 06:59:11.201290 13306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:11.202863 13306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:11.204144 13306 hybrid_clock.cc:648] HybridClock initialized: now 1782025151204092 us; error 48 us; skew 500 ppm
I20260621 06:59:11.204500 13306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:11 dist-test-slave-9zdv krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025151, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:11.210639 13306 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:59:11.212225 13306 webserver.cc:460] Webserver started at http://127.2.245.254:45803/ using document root <none> and password file <none>
I20260621 06:59:11.212940 13306 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:11.213042 13306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:11.213297 13306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:11.216379 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "5a1780a51a874ac18988c6df9852662c"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "cc055348e973443c07adb498610b472b"
server_key_iv: "ecb6885fcc8ebd080a835b657d8d8428"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.217090 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "5a1780a51a874ac18988c6df9852662c"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "cc055348e973443c07adb498610b472b"
server_key_iv: "ecb6885fcc8ebd080a835b657d8d8428"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.222191 13306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 06:59:11.225657 13321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:11.227124 13306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:59:11.227321 13306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "5a1780a51a874ac18988c6df9852662c"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "cc055348e973443c07adb498610b472b"
server_key_iv: "ecb6885fcc8ebd080a835b657d8d8428"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.227514 13306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:11.249465 13306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:11.253499 13306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:11.253780 13306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:11.272686 13306 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:34423
I20260621 06:59:11.272720 13372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:34423 every 8 connection(s)
I20260621 06:59:11.274073 13306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:59:11.274654 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13306
I20260621 06:59:11.274889 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:59:11.275724 3031 external_mini_cluster.cc:1428] Setting key e62f7962c3596e162d879eb24b216d01
I20260621 06:59:11.278509 13373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 06:59:11 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:11.290272 13373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: Bootstrap starting.
I20260621 06:59:11.293767 13373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:11.294824 13373 log.cc:824] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:11.298125 13373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: No bootstrap required, opened a new log
I20260621 06:59:11.298632 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:11.277749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.2.245.254:34423)
0621 06:59:11.278539 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:11.278575 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:11.278634 (+ 59us) server_negotiation.cc:407] Connection header received
0621 06:59:11.280321 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:11.280343 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:11.280713 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:11.281120 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:11.282608 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.283972 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.285358 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.286034 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.286282 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:11.291095 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:11.291104 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:11.291119 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:11.291154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:11.294341 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:11.295115 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:11.295123 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:11.295129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:11.295304 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:11.295812 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:11.295820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:11.295826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:11.296487 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:11.296713 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:11.297161 (+ 448us) server_negotiation.cc:299] Negotiation successful
0621 06:59:11.297569 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":341,"threads_started":1}
I20260621 06:59:11.306234 13373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } }
I20260621 06:59:11.306643 13373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:11.306756 13373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1780a51a874ac18988c6df9852662c, State: Initialized, Role: FOLLOWER
I20260621 06:59:11.307367 13373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } }
I20260621 06:59:11.307608 13373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:59:11.307747 13373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:59:11.307963 13373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:11.310144 13373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } }
I20260621 06:59:11.310592 13373 leader_election.cc:302] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a1780a51a874ac18988c6df9852662c; no voters:
I20260621 06:59:11.311254 13373 leader_election.cc:288] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:59:11.311411 13378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:59:11.312160 13378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [term 1 LEADER]: Becoming Leader. State: Replica: 5a1780a51a874ac18988c6df9852662c, State: Running, Role: LEADER
I20260621 06:59:11.312836 13378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } }
I20260621 06:59:11.313604 13373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:59:11.316076 13379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a1780a51a874ac18988c6df9852662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } } }
I20260621 06:59:11.316098 13380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a1780a51a874ac18988c6df9852662c. Latest consensus state: current_term: 1 leader_uuid: "5a1780a51a874ac18988c6df9852662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1780a51a874ac18988c6df9852662c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 34423 } } }
I20260621 06:59:11.316474 13379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:11.316474 13380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:11.318169 13384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:59:11.323933 13384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:59:11.333225 13384 catalog_manager.cc:1347] Generated new cluster ID: f12769bc12d449fb90d8d5108cd43765
I20260621 06:59:11.333415 13384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:59:11.346112 13384 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:59:11.347689 13384 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:59:11.356072 13384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: Generated new TSK 0
I20260621 06:59:11.357197 13384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:59:11.420858 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:59:11.754742 13399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:11.755087 13399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:11.755234 13399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:11.763893 13399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:11.764009 13399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:11.764189 13399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:59:11.774037 13399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:11.775413 13399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:11.776696 13399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:11.789798 13404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:11.789803 13405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:11.789840 13407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:11.790593 13399 server_base.cc:1034] running on GCE node
I20260621 06:59:11.791241 13399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:11.792135 13399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:11.793383 13399 hybrid_clock.cc:648] HybridClock initialized: now 1782025151793323 us; error 54 us; skew 500 ppm
I20260621 06:59:11.793689 13399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:11 dist-test-slave-9zdv krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025151, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:11.796847 13399 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:59:11.798497 13399 webserver.cc:460] Webserver started at http://127.2.245.193:46103/ using document root <none> and password file <none>
I20260621 06:59:11.799304 13399 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:11.799407 13399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:11.799649 13399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:11.802397 13399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "7e115a7567f64f2ba3c0901035b86273"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "95bcb86f009bd8673508332ee3b32f07"
server_key_iv: "9fe2708b5572ecd640c1c05b08786b8d"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.803222 13399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "7e115a7567f64f2ba3c0901035b86273"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "95bcb86f009bd8673508332ee3b32f07"
server_key_iv: "9fe2708b5572ecd640c1c05b08786b8d"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.808303 13399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 06:59:11.811630 13414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:11.813037 13399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 06:59:11.813205 13399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "7e115a7567f64f2ba3c0901035b86273"
format_stamp: "Formatted at 2026-06-21 06:59:11 on dist-test-slave-9zdv"
server_key: "95bcb86f009bd8673508332ee3b32f07"
server_key_iv: "9fe2708b5572ecd640c1c05b08786b8d"
server_key_version: "encryptionkey@0"
I20260621 06:59:11.813474 13399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:11.828934 13399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:11.833134 13399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:11.833423 13399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:11.834338 13399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:11.835712 13399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:11.835803 13399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:11.835956 13399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:11.836021 13399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:11.868479 13399 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:39111
I20260621 06:59:11.868515 13526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:39111 every 8 connection(s)
I20260621 06:59:11.869797 13399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:59:11.874256 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13399
I20260621 06:59:11.874514 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:59:11.875159 3031 external_mini_cluster.cc:1428] Setting key bf9692452ab1f24d1f221904c999052d
Jun 21 06:59:11 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025151, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:11.890323 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:11.872287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:38761 (local address 127.2.245.254:34423)
0621 06:59:11.872703 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:11.872707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:11.873663 (+ 956us) server_negotiation.cc:407] Connection header received
0621 06:59:11.875669 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:11.875675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:11.875760 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:11.875860 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:11.877800 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.878493 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.880132 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.880530 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.880680 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:11.884448 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:11.884452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:11.884455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:11.884486 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:11.887131 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:11.888080 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:11.888084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:11.888087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:11.888195 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:11.888678 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:11.888681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:11.888683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:11.888923 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:11.889028 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:11.889906 (+ 878us) server_negotiation.cc:299] Negotiation successful
0621 06:59:11.890066 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":209,"threads_started":1}
I20260621 06:59:11.891841 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:11.872853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34423 (local address 127.2.245.193:38761)
0621 06:59:11.873513 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:11.873553 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:11.875393 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:11.876272 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:11.876283 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:11.877038 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:11.877638 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:11.877655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.878642 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.878650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:11.879979 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:11.879991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:11.880764 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:11.880773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:11.881007 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:11.881725 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:11.881748 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:11.884133 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:11.887289 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:11.887301 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:11.887321 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:11.887956 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:11.888305 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:11.888313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:11.888319 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:11.888566 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:11.889044 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:11.889054 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:11.889511 (+ 457us) client_negotiation.cc:769] Sending connection context
0621 06:59:11.889816 (+ 305us) client_negotiation.cc:240] Negotiation successful
0621 06:59:11.890364 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":251,"threads_started":1}
I20260621 06:59:11.893483 13527 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34423
I20260621 06:59:11.893806 13527 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:11.894533 13527 heartbeater.cc:511] Master 127.2.245.254:34423 requested a full tablet report, sending...
I20260621 06:59:11.896600 13338 ts_manager.cc:194] Registered new tserver with Master: 7e115a7567f64f2ba3c0901035b86273 (127.2.245.193:39111)
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 06:59:11.898090 13338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:38761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:59:11.925810 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:59:12.251493 13533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:12.251762 13533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:12.251855 13533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:12.259128 13533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:12.259234 13533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:12.259348 13533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:59:12.268982 13533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:12.270174 13533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:12.271488 13533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:12.284466 13539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:12.284531 13538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:12.285208 13541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:12.285329 13533 server_base.cc:1034] running on GCE node
I20260621 06:59:12.286224 13533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:12.287297 13533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:12.288522 13533 hybrid_clock.cc:648] HybridClock initialized: now 1782025152288473 us; error 42 us; skew 500 ppm
I20260621 06:59:12.288847 13533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:12 dist-test-slave-9zdv krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025152, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:12.291910 13533 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:59:12.293398 13533 webserver.cc:460] Webserver started at http://127.2.245.194:36715/ using document root <none> and password file <none>
I20260621 06:59:12.294068 13533 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:12.294201 13533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:12.294484 13533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:12.297523 13533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "0405acc3f1d843abb3f8b32c4a6bed8b"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "22f26b2f0cbf500c6188479e7b5eb6eb"
server_key_iv: "4b6d5ca811e21456fc63bd793bd78714"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.298274 13533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "0405acc3f1d843abb3f8b32c4a6bed8b"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "22f26b2f0cbf500c6188479e7b5eb6eb"
server_key_iv: "4b6d5ca811e21456fc63bd793bd78714"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.303330 13533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 06:59:12.306684 13548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.308000 13533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 06:59:12.308171 13533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "0405acc3f1d843abb3f8b32c4a6bed8b"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "22f26b2f0cbf500c6188479e7b5eb6eb"
server_key_iv: "4b6d5ca811e21456fc63bd793bd78714"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.308367 13533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:12.325413 13533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:12.329649 13533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:12.329972 13533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:12.330993 13533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:12.332477 13533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:12.332574 13533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.332695 13533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:12.332775 13533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.365813 13533 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42951
I20260621 06:59:12.365861 13660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42951 every 8 connection(s)
I20260621 06:59:12.367198 13533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:59:12.367700 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13533
I20260621 06:59:12.367921 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:59:12.368537 3031 external_mini_cluster.cc:1428] Setting key 08d8410526957a264ba26db451749cc1
Jun 21 06:59:12 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025152, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:12.388055 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.370177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:48125 (local address 127.2.245.254:34423)
0621 06:59:12.370404 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.370409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.371919 (+ 1510us) server_negotiation.cc:407] Connection header received
0621 06:59:12.373691 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.373696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.373777 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.373910 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:12.375507 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.376209 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.378205 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.378645 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.378749 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.382160 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:12.382164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:12.382167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:12.382196 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:12.384920 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.385636 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.385641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.385645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.385780 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.386228 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.386233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.386235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.386489 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:12.386643 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.387486 (+ 843us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.387715 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260621 06:59:12.389110 13663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.370917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34423 (local address 127.2.245.194:48125)
0621 06:59:12.371769 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:12.371807 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:12.373420 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:12.374167 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:12.374175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:12.374701 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:12.375339 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.375356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.376377 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.376382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.378043 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.378060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.378849 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.378853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.379130 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.379820 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:12.379845 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:12.381944 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:12.385061 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:12.385070 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:12.385088 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:12.385505 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:12.385902 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:12.385906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:12.385910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:12.386089 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:12.386618 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:12.386625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:12.387116 (+ 491us) client_negotiation.cc:769] Sending connection context
0621 06:59:12.387381 (+ 265us) client_negotiation.cc:240] Negotiation successful
0621 06:59:12.387903 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":366,"threads_started":1}
I20260621 06:59:12.390583 13661 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34423
I20260621 06:59:12.390882 13661 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 06:59:12.391788 13661 heartbeater.cc:511] Master 127.2.245.254:34423 requested a full tablet report, sending...
I20260621 06:59:12.393456 13338 ts_manager.cc:194] Registered new tserver with Master: 0405acc3f1d843abb3f8b32c4a6bed8b (127.2.245.194:42951)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
I20260621 06:59:12.394392 13338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:48125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:59:12.420019 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:59:12.737450 13666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:12.737726 13666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:12.737818 13666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:12.744724 13666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:12.744824 13666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:12.744932 13666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 06:59:12.754079 13666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:34423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:12.755270 13666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:12.756479 13666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:12.769351 13674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:12.769418 13671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:12.769452 13672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:12.770367 13666 server_base.cc:1034] running on GCE node
I20260621 06:59:12.771045 13666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:12.771946 13666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:12.773140 13666 hybrid_clock.cc:648] HybridClock initialized: now 1782025152773103 us; error 36 us; skew 500 ppm
I20260621 06:59:12.773458 13666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:12 dist-test-slave-9zdv krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025152, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:12.776554 13666 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 06:59:12.778010 13666 webserver.cc:460] Webserver started at http://127.2.245.195:42899/ using document root <none> and password file <none>
I20260621 06:59:12.778676 13666 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:12.778806 13666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:12.779152 13666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 06:59:12.781955 13666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "c6d3a2120c234ff797051b941250925d"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "09741391c63a4d44a54a8841e43b7f47"
server_key_iv: "0f0e97deaa432c457329b8d28083ca36"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.782696 13666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "c6d3a2120c234ff797051b941250925d"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "09741391c63a4d44a54a8841e43b7f47"
server_key_iv: "0f0e97deaa432c457329b8d28083ca36"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.787854 13666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260621 06:59:12.791333 13681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.792663 13666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 06:59:12.792841 13666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "c6d3a2120c234ff797051b941250925d"
format_stamp: "Formatted at 2026-06-21 06:59:12 on dist-test-slave-9zdv"
server_key: "09741391c63a4d44a54a8841e43b7f47"
server_key_iv: "0f0e97deaa432c457329b8d28083ca36"
server_key_version: "encryptionkey@0"
I20260621 06:59:12.793046 13666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:12.806311 13666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:12.810179 13666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:12.810475 13666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:12.811693 13666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:12.813098 13666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:12.813191 13666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.813326 13666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:12.813398 13666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:12.846961 13666 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:44149
I20260621 06:59:12.847044 13793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:44149 every 8 connection(s)
I20260621 06:59:12.848255 13666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 06:59:12.850957 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 13666
I20260621 06:59:12.851171 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 06:59:12.851719 3031 external_mini_cluster.cc:1428] Setting key 235e39bbec10676e8f60a26bce11556d
Jun 21 06:59:12 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025152, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:12.868567 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.851191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:59951 (local address 127.2.245.254:34423)
0621 06:59:12.851415 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.851421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.852843 (+ 1422us) server_negotiation.cc:407] Connection header received
0621 06:59:12.855003 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.855010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.855089 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.855218 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:12.856678 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.857417 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.859014 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.859398 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.859531 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.862808 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:12.862812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:12.862815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:12.862862 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:12.865191 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.866046 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.866053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.866057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.866189 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.866689 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.866694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.866698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.867076 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:12.867207 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.868017 (+ 810us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.868238 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 06:59:12.869551 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.851787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:34423 (local address 127.2.245.195:59951)
0621 06:59:12.852686 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:12.852728 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:12.854693 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:12.855449 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:12.855457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:12.855962 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:12.856503 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.856520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.857562 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.857569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.858826 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.858837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.859632 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.859639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.859871 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.860491 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:12.860514 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:12.862589 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:12.865334 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:12.865342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:12.865368 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:12.865952 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:12.866320 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:12.866324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:12.866328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:12.866592 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:12.867259 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:12.867268 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:12.867597 (+ 329us) client_negotiation.cc:769] Sending connection context
0621 06:59:12.867871 (+ 274us) client_negotiation.cc:240] Negotiation successful
0621 06:59:12.868369 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":314,"threads_started":1}
I20260621 06:59:12.870905 13794 heartbeater.cc:347] Connected to a master server at 127.2.245.254:34423
I20260621 06:59:12.871228 13794 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:12.872071 13794 heartbeater.cc:511] Master 127.2.245.254:34423 requested a full tablet report, sending...
I20260621 06:59:12.873667 13338 ts_manager.cc:194] Registered new tserver with Master: c6d3a2120c234ff797051b941250925d (127.2.245.195:44149)
I20260621 06:59:12.874473 13338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:59951
I20260621 06:59:12.879349 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 06:59:12.895525 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.883662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.2.245.254:34423)
0621 06:59:12.883890 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.883899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.884253 (+ 354us) server_negotiation.cc:407] Connection header received
0621 06:59:12.884413 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.884417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.884475 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.884556 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:12.885848 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.886574 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.887844 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.888226 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.888320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.890708 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:12.890713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:12.890716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:12.890744 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:12.893133 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.893745 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.893749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.893757 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.893884 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:12.894250 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:12.894253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:12.894255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:12.894537 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:12.894704 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.895015 (+ 311us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.895187 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:59:12.900972 13527 heartbeater.cc:503] Master 127.2.245.254:34423 was elected leader, sending a full tablet report...
I20260621 06:59:12.899644 13338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 06:59:12.907049 13338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 06:59:12.946350 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.919576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.2.245.194:42951)
0621 06:59:12.938095 (+ 18519us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.938107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.938148 (+ 41us) server_negotiation.cc:407] Connection header received
0621 06:59:12.938283 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.938291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.938565 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.939584 (+ 1019us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:59:12.939818 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.941374 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.944031 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.945156 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.945298 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.945490 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.945816 (+ 326us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.945997 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18371,"thread_start_us":18272,"threads_started":1}
I20260621 06:59:12.946377 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.919142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42951 (local address 127.0.0.1:45486)
0621 06:59:12.937841 (+ 18699us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:12.937866 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:12.938024 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:12.938880 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:12.938887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:12.938917 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:59:12.939418 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.939431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.941516 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.941523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.943863 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.943892 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.945357 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.945363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.945550 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.945602 (+ 52us) client_negotiation.cc:769] Sending connection context
0621 06:59:12.945730 (+ 128us) client_negotiation.cc:240] Negotiation successful
0621 06:59:12.945934 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18507,"thread_start_us":19131,"threads_started":1}
I20260621 06:59:12.947337 13806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.918997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39460)
0621 06:59:12.934232 (+ 15235us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:12.934290 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:12.934528 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:12.940186 (+ 5658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:12.940193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:12.940240 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:59:12.940672 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.940680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.942257 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.942262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.944653 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.944667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.946464 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.946473 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.946639 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.946671 (+ 32us) client_negotiation.cc:769] Sending connection context
0621 06:59:12.946737 (+ 66us) client_negotiation.cc:240] Negotiation successful
0621 06:59:12.946854 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14863,"mutex_wait_us":112,"spinlock_wait_cycles":1280,"thread_start_us":15765,"threads_started":1}
I20260621 06:59:12.947907 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.920126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.2.245.195:44149)
0621 06:59:12.937246 (+ 17120us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.937259 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.939268 (+ 2009us) server_negotiation.cc:407] Connection header received
0621 06:59:12.939381 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.939390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.939623 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.939808 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:59:12.940794 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.942004 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.944839 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.946195 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.946347 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.946586 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.947429 (+ 843us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.947581 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16899,"thread_start_us":16723,"threads_started":1}
I20260621 06:59:12.948280 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.919103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.2.245.193:39111)
0621 06:59:12.939690 (+ 20587us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:12.939697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:12.939722 (+ 25us) server_negotiation.cc:407] Connection header received
0621 06:59:12.939813 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:12.939820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:12.940000 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:12.940198 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:59:12.940870 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.942034 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.944858 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.946171 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.947571 (+ 1400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.947783 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:12.947844 (+ 61us) server_negotiation.cc:299] Negotiation successful
0621 06:59:12.947985 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20085,"thread_start_us":19935,"threads_started":1}
I20260621 06:59:12.948280 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:12.919562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:44149 (local address 127.0.0.1:33588)
0621 06:59:12.938591 (+ 19029us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:12.938611 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:12.938740 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:12.940072 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:12.940076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:12.940098 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:59:12.940557 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.940569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.942151 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.942162 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.944653 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:12.944667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:12.946727 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:12.946733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:12.946896 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:12.946943 (+ 47us) client_negotiation.cc:769] Sending connection context
0621 06:59:12.947062 (+ 119us) client_negotiation.cc:240] Negotiation successful
0621 06:59:12.947179 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18881,"mutex_wait_us":48,"thread_start_us":17920,"threads_started":1}
I20260621 06:59:12.950893 13596 tablet_service.cc:1467] Processing CreateTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d (DEFAULT_TABLE table=test-table [id=67b270a37e9e4cd69cd51b776994ff37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:59:12.951362 13729 tablet_service.cc:1467] Processing CreateTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d (DEFAULT_TABLE table=test-table [id=67b270a37e9e4cd69cd51b776994ff37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:59:12.952414 13596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac45cb4c78d441cbc44e6b9ab17133d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:12.952688 13729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac45cb4c78d441cbc44e6b9ab17133d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:12.953384 13462 tablet_service.cc:1467] Processing CreateTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d (DEFAULT_TABLE table=test-table [id=67b270a37e9e4cd69cd51b776994ff37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 06:59:12.954620 13462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac45cb4c78d441cbc44e6b9ab17133d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:12.967144 13813 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Bootstrap starting.
I20260621 06:59:12.967935 13812 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Bootstrap starting.
I20260621 06:59:12.969563 13814 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Bootstrap starting.
I20260621 06:59:12.970894 13813 tablet_bootstrap.cc:654] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:12.971899 13813 log.cc:824] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:12.973047 13814 tablet_bootstrap.cc:654] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:12.973047 13812 tablet_bootstrap.cc:654] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:12.974252 13812 log.cc:824] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:12.974251 13814 log.cc:824] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:12.974587 13813 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: No bootstrap required, opened a new log
I20260621 06:59:12.974874 13813 ts_tablet_manager.cc:1397] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260621 06:59:12.981696 13814 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: No bootstrap required, opened a new log
I20260621 06:59:12.981976 13814 ts_tablet_manager.cc:1397] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260621 06:59:12.982298 13812 tablet_bootstrap.cc:492] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: No bootstrap required, opened a new log
I20260621 06:59:12.982563 13812 ts_tablet_manager.cc:1397] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260621 06:59:12.982838 13813 raft_consensus.cc:348] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.983250 13813 raft_consensus.cc:374] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:12.983361 13813 raft_consensus.cc:729] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e115a7567f64f2ba3c0901035b86273, State: Initialized, Role: FOLLOWER
I20260621 06:59:12.983969 13813 consensus_queue.cc:260] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.985646 13813 ts_tablet_manager.cc:1428] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260621 06:59:12.991641 13814 raft_consensus.cc:348] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.992072 13814 raft_consensus.cc:374] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:12.992187 13814 raft_consensus.cc:729] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d3a2120c234ff797051b941250925d, State: Initialized, Role: FOLLOWER
I20260621 06:59:12.992803 13814 consensus_queue.cc:260] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.993252 13812 raft_consensus.cc:348] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.993602 13812 raft_consensus.cc:374] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:12.993721 13812 raft_consensus.cc:729] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0405acc3f1d843abb3f8b32c4a6bed8b, State: Initialized, Role: FOLLOWER
I20260621 06:59:12.993887 13794 heartbeater.cc:503] Master 127.2.245.254:34423 was elected leader, sending a full tablet report...
I20260621 06:59:12.994273 13812 consensus_queue.cc:260] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:12.994494 13814 ts_tablet_manager.cc:1428] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
I20260621 06:59:12.995466 13661 heartbeater.cc:503] Master 127.2.245.254:34423 was elected leader, sending a full tablet report...
I20260621 06:59:12.996101 13812 ts_tablet_manager.cc:1428] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260621 06:59:13.048614 13818 raft_consensus.cc:482] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 06:59:13.048956 13818 raft_consensus.cc:504] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:13.050681 13818 leader_election.cc:288] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0405acc3f1d843abb3f8b32c4a6bed8b (127.2.245.194:42951), c6d3a2120c234ff797051b941250925d (127.2.245.195:44149)
I20260621 06:59:13.059782 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.051245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:55523 (local address 127.2.245.194:42951)
0621 06:59:13.051588 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.051593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.051983 (+ 390us) server_negotiation.cc:407] Connection header received
0621 06:59:13.052203 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.052210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.052291 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.052403 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:59:13.053253 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.054381 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.057573 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.058808 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.058999 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.059072 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.059362 (+ 290us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.059514 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260621 06:59:13.059818 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.051033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42951 (local address 127.2.245.193:55523)
0621 06:59:13.051842 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.051870 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:13.052037 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:13.052465 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:13.052470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:13.052497 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:59:13.053098 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:13.053106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.054522 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.054527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:13.057398 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:13.057406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.059057 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.059063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:13.059192 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.059221 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 06:59:13.059300 (+ 79us) client_negotiation.cc:240] Negotiation successful
0621 06:59:13.059409 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":223,"thread_start_us":440,"threads_started":1}
I20260621 06:59:13.060038 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.051170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:44707 (local address 127.2.245.195:44149)
0621 06:59:13.051375 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.051383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.052149 (+ 766us) server_negotiation.cc:407] Connection header received
0621 06:59:13.052266 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.052272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.052346 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.052435 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 06:59:13.053253 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.054380 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.057571 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.058809 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.059005 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.059072 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.059629 (+ 557us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.059789 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 06:59:13.060297 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.051164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:44149 (local address 127.2.245.193:44707)
0621 06:59:13.052020 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.052037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:13.052146 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:13.052465 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:13.052470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:13.052560 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 06:59:13.053072 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:13.053092 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.054523 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.054527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:13.057389 (+ 2862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:13.057405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.059235 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.059241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:13.059445 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.059480 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 06:59:13.059577 (+ 97us) client_negotiation.cc:240] Negotiation successful
0621 06:59:13.059710 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":233,"spinlock_wait_cycles":21760,"thread_start_us":572,"threads_started":1}
I20260621 06:59:13.061094 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac45cb4c78d441cbc44e6b9ab17133d" candidate_uuid: "7e115a7567f64f2ba3c0901035b86273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" is_pre_election: true
I20260621 06:59:13.061247 13749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac45cb4c78d441cbc44e6b9ab17133d" candidate_uuid: "7e115a7567f64f2ba3c0901035b86273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d3a2120c234ff797051b941250925d" is_pre_election: true
I20260621 06:59:13.061622 13616 raft_consensus.cc:2454] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e115a7567f64f2ba3c0901035b86273 in term 0.
I20260621 06:59:13.061690 13749 raft_consensus.cc:2454] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e115a7567f64f2ba3c0901035b86273 in term 0.
I20260621 06:59:13.062340 13417 leader_election.cc:302] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0405acc3f1d843abb3f8b32c4a6bed8b, 7e115a7567f64f2ba3c0901035b86273, c6d3a2120c234ff797051b941250925d; no voters:
I20260621 06:59:13.062733 13818 raft_consensus.cc:2781] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 06:59:13.062898 13818 raft_consensus.cc:482] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 06:59:13.063060 13818 raft_consensus.cc:3037] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:13.065248 13818 raft_consensus.cc:504] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:13.065834 13818 leader_election.cc:288] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [CANDIDATE]: Term 1 election: Requested vote from peers 0405acc3f1d843abb3f8b32c4a6bed8b (127.2.245.194:42951), c6d3a2120c234ff797051b941250925d (127.2.245.195:44149)
I20260621 06:59:13.066357 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac45cb4c78d441cbc44e6b9ab17133d" candidate_uuid: "7e115a7567f64f2ba3c0901035b86273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b"
I20260621 06:59:13.066355 13749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac45cb4c78d441cbc44e6b9ab17133d" candidate_uuid: "7e115a7567f64f2ba3c0901035b86273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d3a2120c234ff797051b941250925d"
I20260621 06:59:13.066562 13616 raft_consensus.cc:3037] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:13.066562 13749 raft_consensus.cc:3037] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:13.069799 13749 raft_consensus.cc:2454] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e115a7567f64f2ba3c0901035b86273 in term 1.
I20260621 06:59:13.069799 13616 raft_consensus.cc:2454] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e115a7567f64f2ba3c0901035b86273 in term 1.
I20260621 06:59:13.070223 13416 leader_election.cc:302] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0405acc3f1d843abb3f8b32c4a6bed8b, 7e115a7567f64f2ba3c0901035b86273, c6d3a2120c234ff797051b941250925d; no voters:
I20260621 06:59:13.070605 13818 raft_consensus.cc:2781] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:59:13.071106 13818 raft_consensus.cc:686] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 1 LEADER]: Becoming Leader. State: Replica: 7e115a7567f64f2ba3c0901035b86273, State: Running, Role: LEADER
I20260621 06:59:13.071635 13818 consensus_queue.cc:237] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } }
I20260621 06:59:13.075920 13336 catalog_manager.cc:5495] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e115a7567f64f2ba3c0901035b86273 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "7e115a7567f64f2ba3c0901035b86273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e115a7567f64f2ba3c0901035b86273" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 } health_report { overall_health: UNKNOWN } } }
W20260621 06:59:13.099994 13795 tablet.cc:2369] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:59:13.100812 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.095203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.2.245.193:39111)
0621 06:59:13.095395 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.095400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.095536 (+ 136us) server_negotiation.cc:407] Connection header received
0621 06:59:13.095706 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.095710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.095799 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.095923 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 06:59:13.096326 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.097028 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.098464 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.098865 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.099026 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.099487 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 06:59:13.100159 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 06:59:13.100315 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.100476 (+ 161us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.100558 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 06:59:13.108436 13749 raft_consensus.cc:1261] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 1 FOLLOWER]: Refusing update from remote peer 7e115a7567f64f2ba3c0901035b86273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:59:13.108511 13616 raft_consensus.cc:1261] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 1 FOLLOWER]: Refusing update from remote peer 7e115a7567f64f2ba3c0901035b86273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 06:59:13.109494 13823 consensus_queue.cc:1035] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0405acc3f1d843abb3f8b32c4a6bed8b" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 06:59:13.109870 13818 consensus_queue.cc:1035] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d3a2120c234ff797051b941250925d" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 44149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 06:59:13.119534 13662 tablet.cc:2369] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 06:59:13.121622 13827 mvcc.cc:205] Tried to move back new op lower bound from 7299175027122774016 to 7299175026985144320. Current Snapshot: MvccSnapshot[applied={T|T < 7299175027122774016}]
I20260621 06:59:13.124752 13828 mvcc.cc:205] Tried to move back new op lower bound from 7299175027122774016 to 7299175026985144320. Current Snapshot: MvccSnapshot[applied={T|T < 7299175027122774016}]
I20260621 06:59:13.138432 13337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:59:13.138749 13337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 06:59:13.142768 13337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a1780a51a874ac18988c6df9852662c: Sending DeleteTablet for 3 replicas of tablet bac45cb4c78d441cbc44e6b9ab17133d
I20260621 06:59:13.144011 13596 tablet_service.cc:1514] Processing DeleteTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:59:13 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:45486
I20260621 06:59:13.144048 13729 tablet_service.cc:1514] Processing DeleteTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:59:13 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33588
I20260621 06:59:13.144014 13462 tablet_service.cc:1514] Processing DeleteTablet for tablet bac45cb4c78d441cbc44e6b9ab17133d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 06:59:13 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:39460
I20260621 06:59:13.146714 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13399
I20260621 06:59:13.156594 13836 tablet_replica.cc:330] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: stopping tablet replica
I20260621 06:59:13.157239 13838 tablet_replica.cc:330] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: stopping tablet replica
I20260621 06:59:13.157915 13836 raft_consensus.cc:2229] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:59:13.158455 13838 raft_consensus.cc:2229] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 06:59:13.159065 13838 raft_consensus.cc:2258] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:59:13.159571 13836 raft_consensus.cc:2258] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:59:13.162328 13838 ts_tablet_manager.cc:1905] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:59:13.163702 13836 ts_tablet_manager.cc:1905] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:59:13.164322 13837 tablet_replica.cc:330] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: stopping tablet replica
I20260621 06:59:13.166577 13837 raft_consensus.cc:2229] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 1 LEADER]: Raft consensus shutting down.
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 06:59:13.171264 13838 ts_tablet_manager.cc:1918] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:59:13.171433 13838 log.cc:1196] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/bac45cb4c78d441cbc44e6b9ab17133d
I20260621 06:59:13.171680 13836 ts_tablet_manager.cc:1918] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 06:59:13.171941 13836 log.cc:1196] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/bac45cb4c78d441cbc44e6b9ab17133d
I20260621 06:59:13.172030 13838 ts_tablet_manager.cc:1939] T bac45cb4c78d441cbc44e6b9ab17133d P 0405acc3f1d843abb3f8b32c4a6bed8b: Deleting consensus metadata
I20260621 06:59:13.172518 13836 ts_tablet_manager.cc:1939] T bac45cb4c78d441cbc44e6b9ab17133d P c6d3a2120c234ff797051b941250925d: Deleting consensus metadata
I20260621 06:59:13.172773 13837 raft_consensus.cc:2258] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 06:59:13.174026 13324 catalog_manager.cc:4841] TS 0405acc3f1d843abb3f8b32c4a6bed8b (127.2.245.194:42951): tablet bac45cb4c78d441cbc44e6b9ab17133d (table test-table [id=67b270a37e9e4cd69cd51b776994ff37]) successfully deleted
I20260621 06:59:13.174886 13323 catalog_manager.cc:4841] TS c6d3a2120c234ff797051b941250925d (127.2.245.195:44149): tablet bac45cb4c78d441cbc44e6b9ab17133d (table test-table [id=67b270a37e9e4cd69cd51b776994ff37]) successfully deleted
I20260621 06:59:13.176378 13837 ts_tablet_manager.cc:1905] T bac45cb4c78d441cbc44e6b9ab17133d P 7e115a7567f64f2ba3c0901035b86273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 06:59:13.179407 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.149208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.2.245.193:39111)
0621 06:59:13.149457 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.149466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.151346 (+ 1880us) server_negotiation.cc:407] Connection header received
0621 06:59:13.157461 (+ 6115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.157471 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.157614 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.157711 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:13.160036 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.161656 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.164634 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.165352 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.165460 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.171796 (+ 6336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:13.171809 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:13.171829 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:13.171882 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:13.176113 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.176995 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.177006 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.177011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.177196 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.177670 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.177677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.177682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.178141 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:13.178455 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.178775 (+ 320us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.179039 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 06:59:13.180048 13522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:13.280073 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13399
I20260621 06:59:13.298436 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13533
W20260621 06:59:13.298444 13325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:39111 (error 108)
I20260621 06:59:13.300375 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.299877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39482)
0621 06:59:13.300033 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.300186 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260621 06:59:13.300616 13325 catalog_manager.cc:4584] TS 7e115a7567f64f2ba3c0901035b86273 (127.2.245.193:39111): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bac45cb4c78d441cbc44e6b9ab17133d: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 06:59:13.313530 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.299266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.2.245.194:42951)
0621 06:59:13.299440 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.299444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.299625 (+ 181us) server_negotiation.cc:407] Connection header received
0621 06:59:13.299901 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.299908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.300003 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.300112 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:13.301541 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.302234 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.303577 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.303953 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.304057 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.307873 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:13.307887 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:13.307903 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:13.307935 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:13.310775 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.311490 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.311498 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.311500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.311629 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.311981 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.311985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.311987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.312373 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:13.312634 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.312966 (+ 332us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.313162 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 06:59:13.314160 13656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:13.354301 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.353833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39498)
0621 06:59:13.354055 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.354138 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260621 06:59:13.355509 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.355091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39514)
0621 06:59:13.355273 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.355386 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 06:59:13.403263 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13533
I20260621 06:59:13.416018 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.415430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39524)
0621 06:59:13.415677 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.415798 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260621 06:59:13.417188 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.416725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39530)
0621 06:59:13.416915 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.417015 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260621 06:59:13.420651 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13666
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 06:59:13.435976 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.421383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.2.245.195:44149)
0621 06:59:13.421566 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:13.421573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:13.421592 (+ 19us) server_negotiation.cc:407] Connection header received
0621 06:59:13.421807 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:13.421814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:13.421893 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:13.422002 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:13.423299 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.423996 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.425232 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:13.425642 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:13.425782 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:13.430202 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:13.430216 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:13.430231 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:13.430269 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:13.433271 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.433920 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.433927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.433929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.434067 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:13.434451 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:13.434457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:13.434460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:13.434813 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:13.435150 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:13.435446 (+ 296us) server_negotiation.cc:299] Negotiation successful
0621 06:59:13.435661 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 06:59:13.436642 13789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:13.497961 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.497407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39536)
0621 06:59:13.497656 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.497766 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260621 06:59:13.499147 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:13.498648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39111 (local address 127.0.0.1:39548)
0621 06:59:13.498843 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:13.498986 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260621 06:59:13.525701 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13666
I20260621 06:59:13.543560 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 13306
I20260621 06:59:13.544772 13368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 06:59:13.653590 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 13306
2026-06-21T06:59:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4174 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13847](info): set up 2 sockets
Jun 21 06:59:13 dist-test-slave-9zdv krb5kdc[13847](info): commencing operation
krb5kdc: starting...
W20260621 06:59:14.751927 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 06:59:14 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T06:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T06:59:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangeradmin_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangeradmin_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangerlookup_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangerlookup_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
I20260621 06:59:15.079947 3031 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/postgres -l logfile start
2026-06-21 06:59:17.449 UTC [13877] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-21 06:59:17.449 UTC [13877] LOG: listening on IPv4 address "127.2.245.212", port 39155
2026-06-21 06:59:17.452 UTC [13877] LOG: listening on Unix socket "/tmp/.s.PGSQL.39155"
2026-06-21 06:59:17.472 UTC [13880] LOG: database system was shut down at 2026-06-21 06:59:16 UTC
2026-06-21 06:59:17.475 UTC [13877] LOG: database system is ready to accept connections
I20260621 06:59:18.443742 3031 mini_postgres.cc:96] Postgres bound to 39155
2026-06-21 06:59:18.461 UTC [13888] FATAL: database "slave" does not exist
127.2.245.212:39155 - accepting connections
I20260621 06:59:18.461871 3031 mini_ranger.cc:161] Starting Ranger...
I20260621 06:59:18.488129 3031 mini_ranger.cc:84] Created miniranger Postgres user
I20260621 06:59:18.942405 3031 mini_ranger.cc:87] Created ranger Postgres database
I20260621 06:59:18.942552 3031 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin
2026-06-21 06:59:19,233 [I] DB FLAVOR :POSTGRES
2026-06-21 06:59:19,234 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 06:59:19,234 [I] Checking connection..
2026-06-21 06:59:19,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-21 06:59:19,513 [I] Checking connection passed.
2026-06-21 06:59:19,513 [I] --------- Verifying version history table ---------
2026-06-21 06:59:19,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 06:59:19,775 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 06:59:19,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 06:59:20,037 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 06:59:20,037 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-21 06:59:20,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-21 06:59:20.259 UTC [13952] WARNING: there is no transaction in progress
2026-06-21 06:59:20,265 [I] create_dbversion_catalog.sql file imported successfully
2026-06-21 06:59:20,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 06:59:20,499 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-21 06:59:20,499 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 06:59:20,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-21 06:59:20,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-21 06:59:20,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9zdv.c.gcp-upstream.internal','N') ;"
2026-06-21 06:59:21,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 06:59:21,471 [I] Table x_portal_user does not exist in database ranger
2026-06-21 06:59:21,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 06:59:21,725 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-21 06:59:21,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 06:59:21,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 06:59:22,190 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-21 06:59:22,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-21 06:59:23.083 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.099 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.115 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.129 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.144 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.164 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.216 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.222 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.236 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.246 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.256 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.269 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.278 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.284 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.290 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23.297 UTC [14060] WARNING: there is no transaction in progress
2026-06-21 06:59:23,905 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-21 06:59:23,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-21 06:59:24,134 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-21 06:59:24,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 06:59:24,393 [I] Table x_portal_user already exists in database 'ranger'
2026-06-21 06:59:24,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 06:59:24,641 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-21 06:59:24,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 06:59:24,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 06:59:25,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9zdv.c.gcp-upstream.internal';"
2026-06-21 06:59:25,357 [I] CORE_DB_SCHEMA import status has been updated
2026-06-21 06:59:25,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-21 06:59:25,606 [I] DB_PATCHES have already been applied
I20260621 06:59:25.612417 3031 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskV0VCJ_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260621 06:59:25.612502 3031 mini_ranger.cc:193] Using host: 127.2.245.212
I20260621 06:59:25.627185 3031 mini_ranger.cc:227] Ranger admin URL: http://127.2.245.212:50679
Jun 21, 2026 6:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 6:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 6:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp
Jun 21, 2026 6:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 21, 2026 6:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 21, 2026 6:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp].
Jun 21, 2026 6:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260621 06:59:26.778528 3031 mini_ranger.cc:160] Time spent starting Ranger: real 8.317s user 0.011s sys 0.027s
Jun 21, 2026 6:59:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 21, 2026 6:59:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 21, 2026 6:59:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 21, 2026 6:59:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 21, 2026 6:59:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260621 06:59:47.578434 3031 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangerkms_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/rangerkms_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.245.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.245.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 21 06:59:47 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260621 06:59:47.756070 3031 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260621 06:59:47.776882 3031 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260621 06:59:48.605111 3031 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260621 06:59:48.605278 3031 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms
I20260621 06:59:48.605423 3031 mini_ranger_kms.cc:227] Using postgres at 127.2.245.212:39155
2026-06-21 06:59:49,025 [I] DB FLAVOR :POSTGRES
2026-06-21 06:59:49,025 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 06:59:49,025 [I] Checking connection
2026-06-21 06:59:49,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-21 06:59:49,278 [I] connection success
2026-06-21 06:59:49,278 [I] --------- Verifying Ranger DB tables ---------
2026-06-21 06:59:49,279 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-21 06:59:49,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-21 06:59:49,583 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-21 06:59:49,583 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 06:59:49,583 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-21 06:59:49,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:39155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-21 06:59:49,877 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260621 06:59:50.311825 3031 mini_ranger_kms.cc:310] Created kms service
I20260621 06:59:50.531795 3031 mini_ranger_kms.cc:326] Created kudu user
I20260621 06:59:50.579051 3031 mini_ranger_kms.cc:343] Created rangerkms user
Jun 21 06:59:50 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 06:59:50.969358 3031 mini_ranger_kms.cc:384] Added ranger policy
I20260621 06:59:50.969707 3031 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms:/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260621 06:59:50.969771 3031 mini_ranger_kms.cc:240] Using host: 127.2.245.212
I20260621 06:59:50.986864 3031 mini_ranger_kms.cc:276] Ranger KMS PID: 14240
I20260621 06:59:50.987066 3031 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.245.212:57473
Jun 21, 2026 6:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 6:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 6:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 21, 2026 6:59:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 21, 2026 6:59:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp].
Jun 21, 2026 6:59:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260621 06:59:53.331518 3031 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.575s user 0.110s sys 0.324s
I20260621 06:59:53.331786 3031 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260621 06:59:53.332015 3031 mini_ranger_kms.cc:401] 127.2.245.212:57473/kms/v1/keys
Jun 21 06:59:53 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782025193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 21 06:59:54 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Jun 21 06:59:54 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025193, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Jun 21 06:59:56 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 06:59:56.824507 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:41845
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:41845
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:57473/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 06:59:57.155793 14291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:57.156054 14291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:57.156213 14291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:57.163678 14291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 06:59:57.163782 14291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:57.163862 14291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:57.163960 14291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 06:59:57.164058 14291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 06:59:57.177589 14291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.245.212:57473/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:41845
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:57.179042 14291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:57.180375 14291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:57.192209 14296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:57.192236 14297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:57.192296 14299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:57.192874 14291 server_base.cc:1034] running on GCE node
I20260621 06:59:57.194021 14291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:57.195463 14291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:57.196664 14291 hybrid_clock.cc:648] HybridClock initialized: now 1782025197196613 us; error 48 us; skew 500 ppm
I20260621 06:59:57.197033 14291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:57 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025197, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:57.204422 14291 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 06:59:57.205814 14291 webserver.cc:460] Webserver started at http://127.2.245.254:42415/ using document root <none> and password file <none>
I20260621 06:59:57.206436 14291 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:57.206561 14291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:57.206873 14291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 06:59:57 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025197, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 06:59:57.271562 14291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "752d75e975cc4412a90c346b24e2ce5c"
format_stamp: "Formatted at 2026-06-21 06:59:57 on dist-test-slave-9zdv"
server_key: "c7c58b2a9f05a6b0aff5549c2f018c1a"
server_key_iv: "05f0bf2f50c8bacc25b9d874f2ab9d00"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:57.272701 14291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "752d75e975cc4412a90c346b24e2ce5c"
format_stamp: "Formatted at 2026-06-21 06:59:57 on dist-test-slave-9zdv"
server_key: "c7c58b2a9f05a6b0aff5549c2f018c1a"
server_key_iv: "05f0bf2f50c8bacc25b9d874f2ab9d00"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:57.279634 14291 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260621 06:59:57.299883 14306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:57.301613 14291 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260621 06:59:57.301823 14291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "752d75e975cc4412a90c346b24e2ce5c"
format_stamp: "Formatted at 2026-06-21 06:59:57 on dist-test-slave-9zdv"
server_key: "c7c58b2a9f05a6b0aff5549c2f018c1a"
server_key_iv: "05f0bf2f50c8bacc25b9d874f2ab9d00"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:57.302070 14291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:57.346724 14291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:57.351603 14291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:57.351920 14291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:57.372444 14357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:41845 every 8 connection(s)
I20260621 06:59:57.372445 14291 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:41845
I20260621 06:59:57.374197 14291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 06:59:57.378001 14358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 06:59:57.380831 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 14291
I20260621 06:59:57.381031 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 06:59:57.381803 3031 external_mini_cluster.cc:1428] Setting key edefa100b52f8c9a85df7eb6052ba630
I20260621 06:59:57.394054 14358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: Bootstrap starting.
I20260621 06:59:57.398144 14358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: Neither blocks nor log segments found. Creating new log.
I20260621 06:59:57.399549 14358 log.cc:824] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: Log is configured to *not* fsync() on all Append() calls
I20260621 06:59:57.403343 14358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: No bootstrap required, opened a new log
Jun 21 06:59:57 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:57.412859 14358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } }
I20260621 06:59:57.413244 14358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 06:59:57.413378 14358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752d75e975cc4412a90c346b24e2ce5c, State: Initialized, Role: FOLLOWER
I20260621 06:59:57.414034 14358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } }
I20260621 06:59:57.414281 14358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 06:59:57.414426 14358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 06:59:57.414623 14358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 0 FOLLOWER]: Advancing to term 1
I20260621 06:59:57.417330 14358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } }
I20260621 06:59:57.417831 14358 leader_election.cc:302] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 752d75e975cc4412a90c346b24e2ce5c; no voters:
I20260621 06:59:57.418501 14358 leader_election.cc:288] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 06:59:57.418627 14363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 06:59:57.420308 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:57.383950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.2.245.254:41845)
0621 06:59:57.395569 (+ 11619us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:57.395586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:57.396757 (+ 1171us) server_negotiation.cc:407] Connection header received
0621 06:59:57.399366 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:57.399398 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:57.399910 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:57.400424 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:57.402447 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:57.404557 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:57.406491 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:57.407690 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:57.407890 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:57.412478 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:57.412487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:57.412503 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:57.412548 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:57.415747 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:57.416480 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:57.416486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:57.416492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:57.416727 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:57.417111 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:57.417117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:57.417120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:57.417692 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:57.418014 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:57.418502 (+ 488us) server_negotiation.cc:299] Negotiation successful
0621 06:59:57.419163 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11350,"thread_start_us":11094,"threads_started":1}
I20260621 06:59:57.422308 14358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 06:59:57.422708 14363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [term 1 LEADER]: Becoming Leader. State: Replica: 752d75e975cc4412a90c346b24e2ce5c, State: Running, Role: LEADER
I20260621 06:59:57.423431 14363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } }
I20260621 06:59:57.426554 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 752d75e975cc4412a90c346b24e2ce5c. Latest consensus state: current_term: 1 leader_uuid: "752d75e975cc4412a90c346b24e2ce5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } } }
I20260621 06:59:57.427016 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:57.428004 14364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "752d75e975cc4412a90c346b24e2ce5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752d75e975cc4412a90c346b24e2ce5c" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41845 } } }
I20260621 06:59:57.428323 14364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c [sys.catalog]: This master's current role is: LEADER
I20260621 06:59:57.435514 14373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 06:59:57.440430 14373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 06:59:57.441561 14358 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260621 06:59:57.456543 14373 catalog_manager.cc:1347] Generated new cluster ID: 1be7847977824249b0019565a1fee805
I20260621 06:59:57.456698 14373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 06:59:57.485679 14373 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 06:59:57.487737 14373 catalog_manager.cc:1504] Loading token signing keys...
I20260621 06:59:57.508626 14373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: Generated new TSK 0
I20260621 06:59:57.509755 14373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 06:59:58.280625 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:57473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 06:59:58 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260621 06:59:58.694656 14404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:58.695101 14404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:58.695294 14404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:58.706664 14404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:58.706808 14404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:58.707053 14404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 06:59:58.722661 14404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.245.212:57473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:58.724287 14404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:58.725957 14404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:58.743000 14415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:58.743495 14414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:58.747015 14404 server_base.cc:1034] running on GCE node
W20260621 06:59:58.747695 14417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:58.748518 14404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:58.749748 14404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:58.751066 14404 hybrid_clock.cc:648] HybridClock initialized: now 1782025198750989 us; error 64 us; skew 500 ppm
I20260621 06:59:58.751446 14404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:58 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:58.755585 14404 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 06:59:58.757484 14404 webserver.cc:460] Webserver started at http://127.2.245.193:41011/ using document root <none> and password file <none>
I20260621 06:59:58.758452 14404 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:58.758960 14404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:58.759362 14404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 06:59:58 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 06:59:58.781965 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "d3c89d0a400547d8acccd4c49c339ef2"
format_stamp: "Formatted at 2026-06-21 06:59:58 on dist-test-slave-9zdv"
server_key: "45bf0a18679e1eaffd11fd0587f44b24"
server_key_iv: "6e84b224a3da4e7c0c06b2b4aebaebe1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:58.783118 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "d3c89d0a400547d8acccd4c49c339ef2"
format_stamp: "Formatted at 2026-06-21 06:59:58 on dist-test-slave-9zdv"
server_key: "45bf0a18679e1eaffd11fd0587f44b24"
server_key_iv: "6e84b224a3da4e7c0c06b2b4aebaebe1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:58.796591 14404 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.005s
I20260621 06:59:58.822986 14424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:58.824996 14404 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260621 06:59:58.825163 14404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "d3c89d0a400547d8acccd4c49c339ef2"
format_stamp: "Formatted at 2026-06-21 06:59:58 on dist-test-slave-9zdv"
server_key: "45bf0a18679e1eaffd11fd0587f44b24"
server_key_iv: "6e84b224a3da4e7c0c06b2b4aebaebe1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:58.825409 14404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:58.861874 14404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:58.866392 14404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:58.866693 14404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:58.867882 14404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:58.869484 14404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:58.869589 14404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:58.869746 14404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:58.869838 14404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:58.951804 14404 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:34345
I20260621 06:59:58.951860 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:34345 every 8 connection(s)
I20260621 06:59:58.953377 14404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 06:59:58.959937 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 14404
I20260621 06:59:58.960175 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 06:59:58.961257 3031 external_mini_cluster.cc:1428] Setting key 6f9520324db43485d73bd72fadde610e
Jun 21 06:59:58 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:58.987735 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:58.957644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:39897 (local address 127.2.245.254:41845)
0621 06:59:58.967582 (+ 9938us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:58.967588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:58.967611 (+ 23us) server_negotiation.cc:407] Connection header received
0621 06:59:58.968901 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:58.968907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:58.968997 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:58.969088 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:58.971293 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:58.972394 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:58.974879 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:58.975595 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:58.975700 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:58.980389 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:58.980396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:58.980401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:58.980450 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:58.984020 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:58.984970 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:58.984976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:58.984979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:58.985123 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:58.985649 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:58.985655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:58.985658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:58.986010 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:58.986146 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:58.987111 (+ 965us) server_negotiation.cc:299] Negotiation successful
0621 06:59:58.987357 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9815,"thread_start_us":9715,"threads_started":1}
I20260621 06:59:58.991741 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:58.956720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:41845 (local address 127.2.245.193:39897)
0621 06:59:58.964470 (+ 7750us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:58.964523 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:58.968472 (+ 3949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:58.969523 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:58.969536 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:58.970182 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:58.971084 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:58.971111 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:58.972599 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:58.972605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:58.974689 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:58.974718 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:58.976027 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:58.976035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:58.976296 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:58.977388 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:58.977420 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:58.980128 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:58.984199 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:58.984208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:58.984226 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:58.984807 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:58.985245 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:58.985249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:58.985252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:58.985511 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:58.986217 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:58.986226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:58.986645 (+ 419us) client_negotiation.cc:769] Sending connection context
0621 06:59:58.988293 (+ 1648us) client_negotiation.cc:240] Negotiation successful
0621 06:59:58.988843 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7436,"thread_start_us":7202,"threads_started":1}
I20260621 06:59:58.994994 14537 heartbeater.cc:347] Connected to a master server at 127.2.245.254:41845
I20260621 06:59:58.995301 14537 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:58.996117 14537 heartbeater.cc:511] Master 127.2.245.254:41845 requested a full tablet report, sending...
I20260621 06:59:58.998737 14323 ts_manager.cc:194] Registered new tserver with Master: d3c89d0a400547d8acccd4c49c339ef2 (127.2.245.193:34345)
I20260621 06:59:59.000808 14323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:39897
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Jun 21 06:59:59 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 06:59:59.048635 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:57473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 06:59:59 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
W20260621 06:59:59.532812 14543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 06:59:59.533202 14543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 06:59:59.533355 14543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 06:59:59.544673 14543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 06:59:59.544808 14543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 06:59:59.544986 14543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 06:59:59.560808 14543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.245.212:57473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 06:59:59.562541 14543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 06:59:59.564301 14543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 06:59:59.593567 14551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 06:59:59.595650 14550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:59.599465 14543 server_base.cc:1034] running on GCE node
W20260621 06:59:59.599545 14553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 06:59:59.603834 14543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 06:59:59.609081 14543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 06:59:59.610339 14543 hybrid_clock.cc:648] HybridClock initialized: now 1782025199610264 us; error 71 us; skew 500 ppm
I20260621 06:59:59.610749 14543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 06:59:59 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025199, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 06:59:59.615163 14543 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 06:59:59.618811 14543 webserver.cc:460] Webserver started at http://127.2.245.194:46763/ using document root <none> and password file <none>
I20260621 06:59:59.619746 14543 fs_manager.cc:359] Metadata directory not provided
I20260621 06:59:59.619874 14543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 06:59:59.620204 14543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 06:59:59 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025199, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 06:59:59.637917 14543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "d46de9a6c55145feb728afa5764370b9"
format_stamp: "Formatted at 2026-06-21 06:59:59 on dist-test-slave-9zdv"
server_key: "47c440be17765d5167b1b7e7ed3010eb"
server_key_iv: "0fa45b57236d73c9cdc87602754784d1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:59.639518 14543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "d46de9a6c55145feb728afa5764370b9"
format_stamp: "Formatted at 2026-06-21 06:59:59 on dist-test-slave-9zdv"
server_key: "47c440be17765d5167b1b7e7ed3010eb"
server_key_iv: "0fa45b57236d73c9cdc87602754784d1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:59.646652 14543 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.002s
I20260621 06:59:59.673724 14560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:59.675844 14543 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.011s sys 0.003s
I20260621 06:59:59.676141 14543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "d46de9a6c55145feb728afa5764370b9"
format_stamp: "Formatted at 2026-06-21 06:59:59 on dist-test-slave-9zdv"
server_key: "47c440be17765d5167b1b7e7ed3010eb"
server_key_iv: "0fa45b57236d73c9cdc87602754784d1"
server_key_version: "kuduclusterkey@0"
I20260621 06:59:59.676488 14543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 06:59:59.739930 14543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 06:59:59.744220 14543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 06:59:59.744581 14543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 06:59:59.745606 14543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 06:59:59.747027 14543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 06:59:59.747098 14543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:59.747252 14543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 06:59:59.747304 14543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 06:59:59.793411 14543 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38639
I20260621 06:59:59.793432 14679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38639 every 8 connection(s)
I20260621 06:59:59.794656 14543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 06:59:59.805056 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 14543
I20260621 06:59:59.805224 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 06:59:59.805683 3031 external_mini_cluster.cc:1428] Setting key 6dee6a943d5c777b4d9b9dcdc71a3ac1
Jun 21 06:59:59 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025199, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 06:59:59.815798 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:59.797411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:53563 (local address 127.2.245.254:41845)
0621 06:59:59.797875 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0621 06:59:59.797881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 06:59:59.798980 (+ 1099us) server_negotiation.cc:407] Connection header received
0621 06:59:59.800857 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 06:59:59.800863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 06:59:59.800944 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 06:59:59.801074 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0621 06:59:59.802409 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:59.803519 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:59.805018 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 06:59:59.805525 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 06:59:59.805652 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:59.809781 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 06:59:59.809787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 06:59:59.809794 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 06:59:59.809831 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 06:59:59.812709 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:59.813440 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:59.813445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:59.813449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:59.813578 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 06:59:59.814081 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 06:59:59.814086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 06:59:59.814089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 06:59:59.814353 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 06:59:59.814531 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0621 06:59:59.815213 (+ 682us) server_negotiation.cc:299] Negotiation successful
0621 06:59:59.815424 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":223,"threads_started":1}
I20260621 06:59:59.816928 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 06:59:59.797965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:41845 (local address 127.2.245.194:53563)
0621 06:59:59.798786 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0621 06:59:59.798838 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 06:59:59.800587 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 06:59:59.801271 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 06:59:59.801279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 06:59:59.801706 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0621 06:59:59.802245 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:59.802261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:59.803662 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:59.803666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:59.804855 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 06:59:59.804865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 06:59:59.805738 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 06:59:59.805745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 06:59:59.805980 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 06:59:59.807160 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 06:59:59.807192 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0621 06:59:59.809549 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 06:59:59.812875 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:59.812884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:59.812900 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:59.813315 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:59.813695 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 06:59:59.813699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 06:59:59.813703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 06:59:59.813874 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 06:59:59.814482 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 06:59:59.814490 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 06:59:59.814820 (+ 330us) client_negotiation.cc:769] Sending connection context
0621 06:59:59.815132 (+ 312us) client_negotiation.cc:240] Negotiation successful
0621 06:59:59.815626 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":393,"threads_started":1}
I20260621 06:59:59.818331 14680 heartbeater.cc:347] Connected to a master server at 127.2.245.254:41845
I20260621 06:59:59.818668 14680 heartbeater.cc:464] Registering TS with master...
I20260621 06:59:59.819432 14680 heartbeater.cc:511] Master 127.2.245.254:41845 requested a full tablet report, sending...
I20260621 06:59:59.820856 14323 ts_manager.cc:194] Registered new tserver with Master: d46de9a6c55145feb728afa5764370b9 (127.2.245.194:38639)
I20260621 06:59:59.821636 14323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:53563
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 06:59:59.866500 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:57473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:00:00.004650 14537 heartbeater.cc:503] Master 127.2.245.254:41845 was elected leader, sending a full tablet report...
W20260621 07:00:00.206156 14686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:00.206458 14686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:00.206616 14686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:00.214553 14686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:00.214715 14686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:00.214882 14686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:00:00.224660 14686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.245.212:57473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:00.226047 14686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:00.227388 14686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:00.240576 14691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:00.240553 14692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:00.240655 14694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:00.241377 14686 server_base.cc:1034] running on GCE node
I20260621 07:00:00.241920 14686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:00.242782 14686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:00.244064 14686 hybrid_clock.cc:648] HybridClock initialized: now 1782025200244018 us; error 50 us; skew 500 ppm
I20260621 07:00:00.244416 14686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:00 dist-test-slave-9zdv krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025200, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:00.247454 14686 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:00:00.249269 14686 webserver.cc:460] Webserver started at http://127.2.245.195:35457/ using document root <none> and password file <none>
I20260621 07:00:00.249871 14686 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:00.249976 14686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:00.250234 14686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:00:00 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025200, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:00.266310 14686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "2ed8cf5be5f949b4893014eaaf07f562"
format_stamp: "Formatted at 2026-06-21 07:00:00 on dist-test-slave-9zdv"
server_key: "e2793ad60cf1eb042e595b123d69740d"
server_key_iv: "4d97c07d209eafeac1d9e6e72219439c"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:00.267304 14686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "2ed8cf5be5f949b4893014eaaf07f562"
format_stamp: "Formatted at 2026-06-21 07:00:00 on dist-test-slave-9zdv"
server_key: "e2793ad60cf1eb042e595b123d69740d"
server_key_iv: "4d97c07d209eafeac1d9e6e72219439c"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:00.272420 14686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 07:00:00.289745 14701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:00.291793 14686 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:00:00.291982 14686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "2ed8cf5be5f949b4893014eaaf07f562"
format_stamp: "Formatted at 2026-06-21 07:00:00 on dist-test-slave-9zdv"
server_key: "e2793ad60cf1eb042e595b123d69740d"
server_key_iv: "4d97c07d209eafeac1d9e6e72219439c"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:00.292209 14686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:00.305265 14686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:00.308975 14686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:00.309262 14686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:00.310184 14686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:00.311609 14686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:00.311707 14686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:00.311862 14686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:00.311934 14686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:00.347267 14686 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:35037
I20260621 07:00:00.347309 14813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:35037 every 8 connection(s)
I20260621 07:00:00.348557 14686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:00:00.352945 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 14686
I20260621 07:00:00.353130 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:00:00.353703 3031 external_mini_cluster.cc:1428] Setting key c85310fc26dbc12e0473713817435e27
Jun 21 07:00:00 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025200, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:00.368661 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.351264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:52513 (local address 127.2.245.254:41845)
0621 07:00:00.351727 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.351731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.352660 (+ 929us) server_negotiation.cc:407] Connection header received
0621 07:00:00.354975 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.354984 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.355089 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.355203 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:00.356788 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.357526 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.359110 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.359547 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.359680 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.363174 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:00.363179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:00.363182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:00.363212 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:00.365650 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.366408 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.366412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.366415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.366528 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.366993 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.366999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.367007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.367255 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:00.367455 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.368098 (+ 643us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.368304 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":216,"threads_started":1}
I20260621 07:00:00.369504 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.351771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:41845 (local address 127.2.245.195:52513)
0621 07:00:00.352446 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.352482 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.354619 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.355441 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.355449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.355974 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:00.356629 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.356650 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.357675 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.357685 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.358901 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.358912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.359760 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.359768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.360020 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.360616 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:00.360636 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:00.362909 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:00.365800 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:00.365808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:00.365824 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:00.366250 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:00.366639 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:00.366644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:00.366648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:00.366845 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:00.367428 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:00.367436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:00.367751 (+ 315us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.368000 (+ 249us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.368462 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":259,"threads_started":1}
I20260621 07:00:00.370841 14814 heartbeater.cc:347] Connected to a master server at 127.2.245.254:41845
I20260621 07:00:00.371151 14814 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:00.371850 14814 heartbeater.cc:511] Master 127.2.245.254:41845 requested a full tablet report, sending...
I20260621 07:00:00.373247 14323 ts_manager.cc:194] Registered new tserver with Master: 2ed8cf5be5f949b4893014eaaf07f562 (127.2.245.195:35037)
I20260621 07:00:00.374079 14323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:52513
I20260621 07:00:00.381088 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:00:00.397749 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.385611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.2.245.254:41845)
0621 07:00:00.385829 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.385835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.386090 (+ 255us) server_negotiation.cc:407] Connection header received
0621 07:00:00.386206 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.386210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.386275 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.386348 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:00.387716 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.388391 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.389648 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.390471 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.390564 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.392842 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:00.392847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:00.392850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:00.392889 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:00.395456 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.396122 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.396125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.396128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.396234 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.396547 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.396553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.396556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.396799 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:00.396974 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.397306 (+ 332us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.397450 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260621 07:00:00.401595 14323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:00:00.409157 14323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:00:00.447059 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.421896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38639 (local address 127.0.0.1:38058)
0621 07:00:00.437736 (+ 15840us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.437753 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.437879 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.438796 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.438802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.438821 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:00.440059 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.440084 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.442090 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.442097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.444532 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.444547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.445827 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.445833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.446010 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.446062 (+ 52us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.446203 (+ 141us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.446399 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15728,"mutex_wait_us":28,"spinlock_wait_cycles":660224,"thread_start_us":16444,"threads_started":1}
I20260621 07:00:00.446753 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.422244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.2.245.194:38639)
0621 07:00:00.434233 (+ 11989us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.434245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.438003 (+ 3758us) server_negotiation.cc:407] Connection header received
0621 07:00:00.438102 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.438108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.438334 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.438617 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:00.440315 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.441962 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.444698 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.445613 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.445773 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.446008 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.446254 (+ 246us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.446415 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11751,"thread_start_us":11616,"threads_started":1}
I20260621 07:00:00.448910 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.424953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.2.245.195:35037)
0621 07:00:00.431498 (+ 6545us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.431512 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.439398 (+ 7886us) server_negotiation.cc:407] Connection header received
0621 07:00:00.440845 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.440857 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.441010 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.441214 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:00.441767 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.443249 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.445824 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.447264 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.447382 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.447581 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.448398 (+ 817us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.448560 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6304,"thread_start_us":6128,"threads_started":1}
I20260621 07:00:00.449621 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.424261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:35037 (local address 127.0.0.1:37770)
0621 07:00:00.439260 (+ 14999us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.439279 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.440632 (+ 1353us) spinlock_profiling.cc:243] Waited 891 us on lock 0x61200002c7a8. stack: 00007f49be31c66e 00007f49be31c430 00007f49bd3a35ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f49be3120fd 00007f49be311d1e 00007f49d121b3cd 00007f49d121b36c 00007f49d11e7b3b 00007f49c4d21536 00007f49c4d1fd9e 00007f49c4da7dca 00007f49c4da5f68 00007f49c4dd982f
0621 07:00:00.440660 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.441203 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.441208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.441232 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:00.441593 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.441602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.443490 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.443496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.445640 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.445654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.448020 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.448024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.448191 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.448219 (+ 28us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.449040 (+ 821us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.449166 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14873,"spinlock_wait_cycles":2049920,"thread_start_us":14189,"threads_started":1}
I20260621 07:00:00.449863 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.421940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.2.245.193:34345)
0621 07:00:00.437468 (+ 15528us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.437479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.437514 (+ 35us) server_negotiation.cc:407] Connection header received
0621 07:00:00.437624 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.437632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.437930 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.438119 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:00.440351 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.441952 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.446915 (+ 4963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.448182 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.448351 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.448983 (+ 632us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.449347 (+ 364us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.449520 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15344,"thread_start_us":15185,"threads_started":1}
I20260621 07:00:00.449932 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.421611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38130)
0621 07:00:00.436472 (+ 14861us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.436524 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.436740 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.438661 (+ 1921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.438672 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.438711 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:00.440194 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.440207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.444695 (+ 4488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.444699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.446711 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.446725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.448856 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.448863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.449141 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.449183 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.449277 (+ 94us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.449469 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14527,"mutex_wait_us":185,"spinlock_wait_cycles":8064,"thread_start_us":15881,"threads_started":1}
I20260621 07:00:00.451555 14615 tablet_service.cc:1467] Processing CreateTablet for tablet 112c8fabc9544341847f2bfd09a36426 (DEFAULT_TABLE table=test-table [id=dff26c097be74b408d571b01efebe16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:00.452822 14615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112c8fabc9544341847f2bfd09a36426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:00.453567 14472 tablet_service.cc:1467] Processing CreateTablet for tablet 112c8fabc9544341847f2bfd09a36426 (DEFAULT_TABLE table=test-table [id=dff26c097be74b408d571b01efebe16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:00.453567 14749 tablet_service.cc:1467] Processing CreateTablet for tablet 112c8fabc9544341847f2bfd09a36426 (DEFAULT_TABLE table=test-table [id=dff26c097be74b408d571b01efebe16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:00.454918 14472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112c8fabc9544341847f2bfd09a36426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:00.454918 14749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112c8fabc9544341847f2bfd09a36426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:00.464484 14833 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Bootstrap starting.
I20260621 07:00:00.467868 14833 tablet_bootstrap.cc:654] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:00.468992 14833 log.cc:824] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:00.470391 14835 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Bootstrap starting.
I20260621 07:00:00.472162 14833 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: No bootstrap required, opened a new log
I20260621 07:00:00.472414 14833 ts_tablet_manager.cc:1397] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260621 07:00:00.473928 14835 tablet_bootstrap.cc:654] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:00.475610 14835 log.cc:824] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:00.475644 14836 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Bootstrap starting.
I20260621 07:00:00.479385 14836 tablet_bootstrap.cc:654] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:00.479801 14835 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: No bootstrap required, opened a new log
I20260621 07:00:00.480073 14835 ts_tablet_manager.cc:1397] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260621 07:00:00.480432 14836 log.cc:824] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:00.483111 14836 tablet_bootstrap.cc:492] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: No bootstrap required, opened a new log
I20260621 07:00:00.483386 14836 ts_tablet_manager.cc:1397] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260621 07:00:00.484582 14833 raft_consensus.cc:348] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.485033 14833 raft_consensus.cc:374] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:00.485172 14833 raft_consensus.cc:729] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46de9a6c55145feb728afa5764370b9, State: Initialized, Role: FOLLOWER
I20260621 07:00:00.485826 14833 consensus_queue.cc:260] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.487586 14833 ts_tablet_manager.cc:1428] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260621 07:00:00.488121 14680 heartbeater.cc:503] Master 127.2.245.254:41845 was elected leader, sending a full tablet report...
I20260621 07:00:00.490450 14835 raft_consensus.cc:348] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.490818 14835 raft_consensus.cc:374] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:00.490914 14835 raft_consensus.cc:729] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c89d0a400547d8acccd4c49c339ef2, State: Initialized, Role: FOLLOWER
I20260621 07:00:00.491542 14835 consensus_queue.cc:260] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.492750 14836 raft_consensus.cc:348] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.493136 14836 raft_consensus.cc:374] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:00.493249 14836 raft_consensus.cc:729] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed8cf5be5f949b4893014eaaf07f562, State: Initialized, Role: FOLLOWER
I20260621 07:00:00.493323 14835 ts_tablet_manager.cc:1428] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260621 07:00:00.493817 14836 consensus_queue.cc:260] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.494928 14814 heartbeater.cc:503] Master 127.2.245.254:41845 was elected leader, sending a full tablet report...
I20260621 07:00:00.495612 14836 ts_tablet_manager.cc:1428] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Time spent starting tablet: real 0.012s user 0.003s sys 0.009s
W20260621 07:00:00.546722 14681 tablet.cc:2369] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:00:00.600144 14815 tablet.cc:2369] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:00:00.603046 14840 raft_consensus.cc:482] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:00:00.603358 14840 raft_consensus.cc:504] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.605140 14840 leader_election.cc:288] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d46de9a6c55145feb728afa5764370b9 (127.2.245.194:38639), 2ed8cf5be5f949b4893014eaaf07f562 (127.2.245.195:35037)
I20260621 07:00:00.612674 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.605701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48373 (local address 127.2.245.194:38639)
0621 07:00:00.605908 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.605915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.606582 (+ 667us) server_negotiation.cc:407] Connection header received
0621 07:00:00.607229 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.607237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.607306 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.607384 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:00.608042 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.608786 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.610813 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.611704 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.611863 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.611967 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.612301 (+ 334us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.612428 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 07:00:00.612694 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.605641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48159 (local address 127.2.245.195:35037)
0621 07:00:00.605821 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.605828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.607059 (+ 1231us) server_negotiation.cc:407] Connection header received
0621 07:00:00.607304 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.607308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.607394 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.607478 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:00.608160 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.608822 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.610879 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.611722 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.611881 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.611975 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.612349 (+ 374us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.612465 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:00:00.612850 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.605636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:35037 (local address 127.2.245.193:48159)
0621 07:00:00.606517 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.606554 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.607125 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.607515 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.607519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.607543 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:00.607925 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.607934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.609009 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.609012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.610700 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.610738 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.612087 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.612092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.612199 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.612219 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.612325 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.612455 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"spinlock_wait_cycles":993280,"thread_start_us":577,"threads_started":1}
I20260621 07:00:00.613059 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.605620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38639 (local address 127.2.245.193:48373)
0621 07:00:00.606465 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.606480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:00.607068 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:00.607434 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:00.607439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:00.607475 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:00.607892 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.607905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.608927 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.608934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.610645 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:00.610658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.612008 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.612015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:00.612146 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.612168 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 07:00:00.612274 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 07:00:00.612401 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":34,"spinlock_wait_cycles":768,"thread_start_us":479,"threads_started":1}
I20260621 07:00:00.614063 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112c8fabc9544341847f2bfd09a36426" candidate_uuid: "d3c89d0a400547d8acccd4c49c339ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed8cf5be5f949b4893014eaaf07f562" is_pre_election: true
I20260621 07:00:00.614169 14635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112c8fabc9544341847f2bfd09a36426" candidate_uuid: "d3c89d0a400547d8acccd4c49c339ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46de9a6c55145feb728afa5764370b9" is_pre_election: true
I20260621 07:00:00.614622 14769 raft_consensus.cc:2454] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c89d0a400547d8acccd4c49c339ef2 in term 0.
I20260621 07:00:00.614622 14635 raft_consensus.cc:2454] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c89d0a400547d8acccd4c49c339ef2 in term 0.
I20260621 07:00:00.615346 14426 leader_election.cc:302] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ed8cf5be5f949b4893014eaaf07f562, d3c89d0a400547d8acccd4c49c339ef2, d46de9a6c55145feb728afa5764370b9; no voters:
I20260621 07:00:00.615764 14840 raft_consensus.cc:2781] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:00:00.615926 14840 raft_consensus.cc:482] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:00:00.616030 14840 raft_consensus.cc:3037] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:00.618608 14840 raft_consensus.cc:504] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.619236 14840 leader_election.cc:288] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [CANDIDATE]: Term 1 election: Requested vote from peers d46de9a6c55145feb728afa5764370b9 (127.2.245.194:38639), 2ed8cf5be5f949b4893014eaaf07f562 (127.2.245.195:35037)
I20260621 07:00:00.619730 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112c8fabc9544341847f2bfd09a36426" candidate_uuid: "d3c89d0a400547d8acccd4c49c339ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed8cf5be5f949b4893014eaaf07f562"
I20260621 07:00:00.619730 14635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112c8fabc9544341847f2bfd09a36426" candidate_uuid: "d3c89d0a400547d8acccd4c49c339ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46de9a6c55145feb728afa5764370b9"
I20260621 07:00:00.619956 14769 raft_consensus.cc:3037] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:00.619954 14635 raft_consensus.cc:3037] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:00.623118 14769 raft_consensus.cc:2454] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c89d0a400547d8acccd4c49c339ef2 in term 1.
I20260621 07:00:00.623119 14635 raft_consensus.cc:2454] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c89d0a400547d8acccd4c49c339ef2 in term 1.
I20260621 07:00:00.623553 14426 leader_election.cc:302] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ed8cf5be5f949b4893014eaaf07f562, d3c89d0a400547d8acccd4c49c339ef2, d46de9a6c55145feb728afa5764370b9; no voters:
I20260621 07:00:00.623822 14840 raft_consensus.cc:2781] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:00:00.624250 14840 raft_consensus.cc:686] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 1 LEADER]: Becoming Leader. State: Replica: d3c89d0a400547d8acccd4c49c339ef2, State: Running, Role: LEADER
I20260621 07:00:00.624675 14840 consensus_queue.cc:237] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } }
I20260621 07:00:00.629222 14322 catalog_manager.cc:5495] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c89d0a400547d8acccd4c49c339ef2 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "d3c89d0a400547d8acccd4c49c339ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c89d0a400547d8acccd4c49c339ef2" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 } health_report { overall_health: UNKNOWN } } }
I20260621 07:00:00.685302 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.679129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.2.245.193:34345)
0621 07:00:00.679369 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.679376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.679474 (+ 98us) server_negotiation.cc:407] Connection header received
0621 07:00:00.679735 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.679739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.679812 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.679968 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:00:00.680344 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.681098 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.683092 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.683482 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.683617 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.684173 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:00:00.684668 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:00:00.684829 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.684927 (+ 98us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.685005 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260621 07:00:00.694036 14769 raft_consensus.cc:1261] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 1 FOLLOWER]: Refusing update from remote peer d3c89d0a400547d8acccd4c49c339ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:00.694036 14635 raft_consensus.cc:1261] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 1 FOLLOWER]: Refusing update from remote peer d3c89d0a400547d8acccd4c49c339ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:00.695122 14840 consensus_queue.cc:1035] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d46de9a6c55145feb728afa5764370b9" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:00:00.695474 14844 consensus_queue.cc:1035] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed8cf5be5f949b4893014eaaf07f562" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 35037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:00:00.707027 14847 mvcc.cc:205] Tried to move back new op lower bound from 7299175222031892480 to 7299175221761961984. Current Snapshot: MvccSnapshot[applied={T|T < 7299175222031892480}]
I20260621 07:00:00.708854 14849 mvcc.cc:205] Tried to move back new op lower bound from 7299175222031892480 to 7299175221761961984. Current Snapshot: MvccSnapshot[applied={T|T < 7299175222031892480}]
I20260621 07:00:00.721777 14322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:00:00.722078 14322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:00:00.725814 14322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 752d75e975cc4412a90c346b24e2ce5c: Sending DeleteTablet for 3 replicas of tablet 112c8fabc9544341847f2bfd09a36426
I20260621 07:00:00.726984 14472 tablet_service.cc:1514] Processing DeleteTablet for tablet 112c8fabc9544341847f2bfd09a36426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:00 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:38130
I20260621 07:00:00.727041 14615 tablet_service.cc:1514] Processing DeleteTablet for tablet 112c8fabc9544341847f2bfd09a36426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:00 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:38058
I20260621 07:00:00.728434 14749 tablet_service.cc:1514] Processing DeleteTablet for tablet 112c8fabc9544341847f2bfd09a36426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:00 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37770
I20260621 07:00:00.729843 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 14404
I20260621 07:00:00.740949 14857 tablet_replica.cc:330] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: stopping tablet replica
I20260621 07:00:00.741411 14857 raft_consensus.cc:2229] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:00:00.741915 14857 raft_consensus.cc:2258] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:00.745860 14858 tablet_replica.cc:330] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: stopping tablet replica
I20260621 07:00:00.746109 14856 tablet_replica.cc:330] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: stopping tablet replica
I20260621 07:00:00.746309 14858 raft_consensus.cc:2229] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:00:00.746524 14856 raft_consensus.cc:2229] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:00:00.746812 14858 raft_consensus.cc:2258] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:00.748818 14858 ts_tablet_manager.cc:1905] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:00.749646 14857 ts_tablet_manager.cc:1905] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:00.752138 14856 raft_consensus.cc:2258] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:00.754963 14858 ts_tablet_manager.cc:1918] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:00:00.755122 14858 log.cc:1196] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/112c8fabc9544341847f2bfd09a36426
Jun 21 07:00:00 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:00:00.755612 14858 ts_tablet_manager.cc:1939] T 112c8fabc9544341847f2bfd09a36426 P 2ed8cf5be5f949b4893014eaaf07f562: Deleting consensus metadata
I20260621 07:00:00.756839 14857 ts_tablet_manager.cc:1918] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:00:00.756979 14857 log.cc:1196] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/112c8fabc9544341847f2bfd09a36426
I20260621 07:00:00.757476 14310 catalog_manager.cc:4841] TS 2ed8cf5be5f949b4893014eaaf07f562 (127.2.245.195:35037): tablet 112c8fabc9544341847f2bfd09a36426 (table test-table [id=dff26c097be74b408d571b01efebe16b]) successfully deleted
I20260621 07:00:00.758119 14857 ts_tablet_manager.cc:1939] T 112c8fabc9544341847f2bfd09a36426 P d46de9a6c55145feb728afa5764370b9: Deleting consensus metadata
I20260621 07:00:00.759121 14856 ts_tablet_manager.cc:1905] T 112c8fabc9544341847f2bfd09a36426 P d3c89d0a400547d8acccd4c49c339ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:00.759974 14308 catalog_manager.cc:4841] TS d46de9a6c55145feb728afa5764370b9 (127.2.245.194:38639): tablet 112c8fabc9544341847f2bfd09a36426 (table test-table [id=dff26c097be74b408d571b01efebe16b]) successfully deleted
I20260621 07:00:00.764183 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.731492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.2.245.193:34345)
0621 07:00:00.731712 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.731721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.735907 (+ 4186us) server_negotiation.cc:407] Connection header received
0621 07:00:00.736850 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.736856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.736961 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.737034 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:00.742385 (+ 5351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.743687 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.747463 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.748164 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.748273 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.756558 (+ 8285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:00.756576 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:00.756596 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:00.756653 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:00.760981 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.761757 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.761763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.761765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.761919 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.762332 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.762336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.762338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.762718 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:00.763063 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.763366 (+ 303us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.763633 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 07:00:00.764922 14532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:00.876976 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 14404
W20260621 07:00:00.895040 14309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:34345 (error 108)
I20260621 07:00:00.895234 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 14543
I20260621 07:00:00.896986 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.896566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38160)
0621 07:00:00.896747 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.896852 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260621 07:00:00.897198 14309 catalog_manager.cc:4584] TS d3c89d0a400547d8acccd4c49c339ef2 (127.2.245.193:34345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 112c8fabc9544341847f2bfd09a36426: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Jun 21 07:00:00 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:00:00.910634 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.896132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.2.245.194:38639)
0621 07:00:00.896315 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:00.896323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:00.896345 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:00:00.896554 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:00.896569 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:00.896651 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:00.896772 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:00.898192 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.899450 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.900806 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:00.901190 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:00.901337 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:00.905397 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:00.905410 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:00.905425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:00.905456 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:00.908287 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.908933 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.908940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.908942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.909055 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:00.909406 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:00.909410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:00.909412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:00.909744 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:00.909963 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:00.910208 (+ 245us) server_negotiation.cc:299] Negotiation successful
0621 07:00:00.910362 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:00:00.911422 14675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:00.950922 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.950502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38168)
0621 07:00:00.950682 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.950782 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260621 07:00:00.952209 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:00.951844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38180)
0621 07:00:00.952003 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:00.952093 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260621 07:00:00.999495 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 14543
I20260621 07:00:01.012993 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:01.012271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38194)
0621 07:00:01.012650 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:01.012768 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260621 07:00:01.014147 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:01.013676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38200)
0621 07:00:01.013867 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:01.013961 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260621 07:00:01.016463 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 14686
Jun 21 07:00:01 dist-test-slave-9zdv krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:00:01.031013 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:01.017210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.2.245.195:35037)
0621 07:00:01.017392 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:01.017396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:01.017413 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:00:01.017513 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:01.017517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:01.017577 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:01.017654 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:01.018887 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:01.019734 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:01.020962 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:01.021336 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:01.021431 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:01.025888 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:01.025901 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:01.025918 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:01.025952 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:01.028627 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:01.029223 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:01.029229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:01.029232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:01.029348 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:01.029687 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:01.029691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:01.029692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:01.030015 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:01.030302 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:01.030446 (+ 144us) server_negotiation.cc:299] Negotiation successful
0621 07:00:01.030594 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:00:01.031632 14809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:01.094695 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:01.094103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38214)
0621 07:00:01.094374 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:01.094483 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260621 07:00:01.095831 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:01.095421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34345 (local address 127.0.0.1:38220)
0621 07:00:01.095621 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:01.095705 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260621 07:00:01.119699 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 14686
I20260621 07:00:01.135774 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 14291
I20260621 07:00:01.137025 14353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:01.251971 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 14291
I20260621 07:00:01.274125 3031 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260621 07:00:01.609717 3031 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260621 07:00:01.609974 3031 mini_ranger.cc:66] Stopping Ranger...
I20260621 07:00:01.959117 3031 mini_ranger.cc:68] Stopped Ranger
2026-06-21 07:00:01.959 UTC [13877] LOG: received smart shutdown request
2026-06-21 07:00:01.962 UTC [13877] LOG: background worker "logical replication launcher" (PID 13886) exited with exit code 1
2026-06-21 07:00:01.962 UTC [13881] LOG: shutting down
2026-06-21 07:00:02.121 UTC [13877] LOG: database system is shut down
2026-06-21T07:00:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48586 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:00:02 dist-test-slave-9zdv krb5kdc[14873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:00:02 dist-test-slave-9zdv krb5kdc[14873](info): set up 2 sockets
Jun 21 07:00:02 dist-test-slave-9zdv krb5kdc[14873](info): commencing operation
krb5kdc: starting...
W20260621 07:00:03.335644 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:00:03 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:00:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:00:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangeradmin_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangeradmin_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangerlookup_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangerlookup_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
I20260621 07:00:03.646081 3031 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/postgres -l logfile start
2026-06-21 07:00:05.983 UTC [14903] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-21 07:00:05.983 UTC [14903] LOG: listening on IPv4 address "127.2.245.212", port 36165
2026-06-21 07:00:05.987 UTC [14903] LOG: listening on Unix socket "/tmp/.s.PGSQL.36165"
2026-06-21 07:00:06.008 UTC [14906] LOG: database system was shut down at 2026-06-21 07:00:04 UTC
2026-06-21 07:00:06.011 UTC [14903] LOG: database system is ready to accept connections
I20260621 07:00:06.983325 3031 mini_postgres.cc:96] Postgres bound to 36165
2026-06-21 07:00:07.000 UTC [14914] FATAL: database "slave" does not exist
127.2.245.212:36165 - accepting connections
I20260621 07:00:07.001308 3031 mini_ranger.cc:161] Starting Ranger...
I20260621 07:00:07.026736 3031 mini_ranger.cc:84] Created miniranger Postgres user
I20260621 07:00:07.477808 3031 mini_ranger.cc:87] Created ranger Postgres database
I20260621 07:00:07.477916 3031 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin
2026-06-21 07:00:07,761 [I] DB FLAVOR :POSTGRES
2026-06-21 07:00:07,762 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:00:07,762 [I] Checking connection..
2026-06-21 07:00:07,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-21 07:00:08,020 [I] Checking connection passed.
2026-06-21 07:00:08,020 [I] --------- Verifying version history table ---------
2026-06-21 07:00:08,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:00:08,259 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:00:08,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:00:08,518 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:00:08,519 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-21 07:00:08,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-21 07:00:08.771 UTC [14978] WARNING: there is no transaction in progress
2026-06-21 07:00:08,777 [I] create_dbversion_catalog.sql file imported successfully
2026-06-21 07:00:08,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:00:09,020 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-21 07:00:09,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:00:09,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-21 07:00:09,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-21 07:00:09,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9zdv.c.gcp-upstream.internal','N') ;"
2026-06-21 07:00:09,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:00:09,979 [I] Table x_portal_user does not exist in database ranger
2026-06-21 07:00:09,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:00:10,239 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-21 07:00:10,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:00:10,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:00:10,727 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-21 07:00:10,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-21 07:00:11.674 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.689 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.706 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.720 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.734 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.751 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.803 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.809 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.823 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.832 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.842 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.855 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.864 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.869 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.875 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:11.881 UTC [15086] WARNING: there is no transaction in progress
2026-06-21 07:00:12,462 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-21 07:00:12,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-21 07:00:12,698 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-21 07:00:12,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:00:12,929 [I] Table x_portal_user already exists in database 'ranger'
2026-06-21 07:00:12,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:00:13,171 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-21 07:00:13,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:00:13,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:00:13,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9zdv.c.gcp-upstream.internal';"
2026-06-21 07:00:13,906 [I] CORE_DB_SCHEMA import status has been updated
2026-06-21 07:00:13,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-21 07:00:14,166 [I] DB_PATCHES have already been applied
I20260621 07:00:14.172339 3031 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskV0VCJ_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260621 07:00:14.172427 3031 mini_ranger.cc:193] Using host: 127.2.245.212
I20260621 07:00:14.187474 3031 mini_ranger.cc:227] Ranger admin URL: http://127.2.245.212:52429
Jun 21, 2026 7:00:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:00:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:00:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp
Jun 21, 2026 7:00:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 21, 2026 7:00:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 21, 2026 7:00:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-admin/ews/webapp].
Jun 21, 2026 7:00:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260621 07:00:15.318549 3031 mini_ranger.cc:160] Time spent starting Ranger: real 8.317s user 0.006s sys 0.033s
Jun 21, 2026 7:00:34 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 21, 2026 7:00:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 21, 2026 7:00:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 21, 2026 7:00:34 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 21, 2026 7:00:35 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260621 07:00:35.699232 3031 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.245.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.245.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangerkms_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.245.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/rangerkms_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.245.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.245.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.245.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/HTTP_127.2.245.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 21 07:00:35 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025235, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260621 07:00:35.856524 3031 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260621 07:00:35.876775 3031 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260621 07:00:36.673024 3031 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260621 07:00:36.673169 3031 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms
I20260621 07:00:36.673245 3031 mini_ranger_kms.cc:227] Using postgres at 127.2.245.212:36165
2026-06-21 07:00:37,018 [I] DB FLAVOR :POSTGRES
2026-06-21 07:00:37,019 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:00:37,019 [I] Checking connection
2026-06-21 07:00:37,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-21 07:00:37,257 [I] connection success
2026-06-21 07:00:37,258 [I] --------- Verifying Ranger DB tables ---------
2026-06-21 07:00:37,258 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-21 07:00:37,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-21 07:00:37,511 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-21 07:00:37,511 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:00:37,511 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-21 07:00:37,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.245.212:36165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-21 07:00:37,774 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260621 07:00:38.222386 3031 mini_ranger_kms.cc:310] Created kms service
I20260621 07:00:38.389721 3031 mini_ranger_kms.cc:326] Created kudu user
I20260621 07:00:38.434743 3031 mini_ranger_kms.cc:343] Created rangerkms user
Jun 21 07:00:38 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025235, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:38.859272 3031 mini_ranger_kms.cc:384] Added ranger policy
I20260621 07:00:38.859604 3031 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms:/tmp/dist-test-taskV0VCJ_/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskV0VCJ_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260621 07:00:38.859670 3031 mini_ranger_kms.cc:240] Using host: 127.2.245.212
I20260621 07:00:38.877084 3031 mini_ranger_kms.cc:276] Ranger KMS PID: 15266
I20260621 07:00:38.877206 3031 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.245.212:34967
Jun 21, 2026 7:00:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:00:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:00:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 21, 2026 7:00:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 21, 2026 7:00:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-kms/ews/webapp].
Jun 21, 2026 7:00:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 21 07:00:40 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782025240, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:41.184095 3031 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.328s user 0.070s sys 0.294s
I20260621 07:00:41.184347 3031 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260621 07:00:41.184517 3031 mini_ranger_kms.cc:401] 127.2.245.212:34967/kms/v1/keys
Jun 21 07:00:41 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025240, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Jun 21 07:00:41 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025240, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.245.212@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Jun 21 07:00:43 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:00:43.994532 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35507
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35507
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:00:44.326283 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:44.326596 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:44.326748 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:44.334347 15317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:44.334473 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:44.334585 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:44.334682 15317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:44.334777 15317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:44.345266 15317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.245.212:34967/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35507
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:44.346587 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:44.348078 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:44.366374 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:44.366439 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:44.366869 15317 server_base.cc:1034] running on GCE node
W20260621 07:00:44.367072 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:44.367996 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:44.369606 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:44.370865 15317 hybrid_clock.cc:648] HybridClock initialized: now 1782025244370809 us; error 59 us; skew 500 ppm
I20260621 07:00:44.371254 15317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:44 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:44.374388 15317 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:00:44.376070 15317 webserver.cc:460] Webserver started at http://127.2.245.254:39011/ using document root <none> and password file <none>
I20260621 07:00:44.376740 15317 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:44.376878 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:44.377183 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:00:44 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:44.428718 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "7bb4e8bd315c453c960c499648f5322d"
format_stamp: "Formatted at 2026-06-21 07:00:44 on dist-test-slave-9zdv"
server_key: "772323d1bcb121358484875ff87001e3"
server_key_iv: "3b7d13bd0d0ce2137f8a723061d40e4f"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:44.429708 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "7bb4e8bd315c453c960c499648f5322d"
format_stamp: "Formatted at 2026-06-21 07:00:44 on dist-test-slave-9zdv"
server_key: "772323d1bcb121358484875ff87001e3"
server_key_iv: "3b7d13bd0d0ce2137f8a723061d40e4f"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:44.435377 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260621 07:00:44.455497 15332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:44.457047 15317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:00:44.457221 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "7bb4e8bd315c453c960c499648f5322d"
format_stamp: "Formatted at 2026-06-21 07:00:44 on dist-test-slave-9zdv"
server_key: "772323d1bcb121358484875ff87001e3"
server_key_iv: "3b7d13bd0d0ce2137f8a723061d40e4f"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:44.457479 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:44.480477 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:44.484740 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:44.485096 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:44.504513 15317 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35507
I20260621 07:00:44.504531 15383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35507 every 8 connection(s)
I20260621 07:00:44.505890 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:00:44.509690 15384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:44.511389 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15317
I20260621 07:00:44.511631 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:00:44.512471 3031 external_mini_cluster.cc:1428] Setting key 5d0909fb969b0b1faeaead75d25a2bc9
I20260621 07:00:44.522240 15384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Bootstrap starting.
I20260621 07:00:44.526044 15384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Neither blocks nor log segments found. Creating new log.
Jun 21 07:00:44 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:44.527350 15384 log.cc:824] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:44.530821 15384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: No bootstrap required, opened a new log
I20260621 07:00:44.536231 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:44.514582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53682 (local address 127.2.245.254:35507)
0621 07:00:44.515430 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:44.515446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:44.515493 (+ 47us) server_negotiation.cc:407] Connection header received
0621 07:00:44.517134 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:44.517156 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:44.517539 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:44.517844 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:44.519511 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:44.520822 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:44.522700 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:44.523426 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:44.523634 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:44.528129 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:44.528137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:44.528152 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:44.528186 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:44.531815 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:44.532442 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:44.532451 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:44.532459 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:44.532661 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:44.533017 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:44.533022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:44.533024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:44.533709 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:44.534042 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:44.534411 (+ 369us) server_negotiation.cc:299] Negotiation successful
0621 07:00:44.534853 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":450,"threads_started":1}
I20260621 07:00:44.540565 15384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:44.540921 15384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:44.541033 15384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb4e8bd315c453c960c499648f5322d, State: Initialized, Role: FOLLOWER
I20260621 07:00:44.541603 15384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:44.541857 15384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:00:44.541982 15384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:00:44.542143 15384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:44.544313 15384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:44.544845 15384 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bb4e8bd315c453c960c499648f5322d; no voters:
I20260621 07:00:44.545539 15384 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:00:44.545603 15389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:00:44.546411 15389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 LEADER]: Becoming Leader. State: Replica: 7bb4e8bd315c453c960c499648f5322d, State: Running, Role: LEADER
I20260621 07:00:44.546917 15389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:44.547783 15384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:44.549965 15390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bb4e8bd315c453c960c499648f5322d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } } }
I20260621 07:00:44.550364 15390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:44.550176 15391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bb4e8bd315c453c960c499648f5322d. Latest consensus state: current_term: 1 leader_uuid: "7bb4e8bd315c453c960c499648f5322d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } } }
I20260621 07:00:44.550495 15391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:44.560922 15397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:00:44.565199 15397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:00:44.565932 15384 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260621 07:00:44.584496 15397 catalog_manager.cc:1347] Generated new cluster ID: 82fa33bd116e44b4b2d7df8073758d80
I20260621 07:00:44.584714 15397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:00:44.605993 15397 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:00:44.608377 15397 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:00:44.621855 15397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Generated new TSK 0
I20260621 07:00:44.622985 15397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:00:45.354418 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:00:45 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260621 07:00:45.776350 15429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:45.776788 15429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:45.776986 15429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:45.788486 15429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:45.788682 15429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:45.788913 15429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:00:45.804435 15429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:45.806149 15429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:45.807897 15429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:45.823839 15439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:45.825383 15440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:45.832924 15442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:45.833495 15429 server_base.cc:1034] running on GCE node
I20260621 07:00:45.834245 15429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:45.847290 15429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:45.850775 15429 hybrid_clock.cc:648] HybridClock initialized: now 1782025245850709 us; error 80 us; skew 500 ppm
I20260621 07:00:45.851301 15429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:45 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:45.855791 15429 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:00:45.863395 15429 webserver.cc:460] Webserver started at http://127.2.245.193:39361/ using document root <none> and password file <none>
I20260621 07:00:45.864261 15429 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:45.864419 15429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:45.864830 15429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:00:45 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:45.882982 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "61444532d00e479fa63cfc88f617a5ca"
format_stamp: "Formatted at 2026-06-21 07:00:45 on dist-test-slave-9zdv"
server_key: "2f96568c9bf6372450741d83194fed1e"
server_key_iv: "b393c502709fededc164341bcf818678"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:45.884169 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "61444532d00e479fa63cfc88f617a5ca"
format_stamp: "Formatted at 2026-06-21 07:00:45 on dist-test-slave-9zdv"
server_key: "2f96568c9bf6372450741d83194fed1e"
server_key_iv: "b393c502709fededc164341bcf818678"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:45.909200 15429 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.010s sys 0.012s
I20260621 07:00:45.928409 15449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:45.930533 15429 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:00:45.930773 15429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "61444532d00e479fa63cfc88f617a5ca"
format_stamp: "Formatted at 2026-06-21 07:00:45 on dist-test-slave-9zdv"
server_key: "2f96568c9bf6372450741d83194fed1e"
server_key_iv: "b393c502709fededc164341bcf818678"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:45.931103 15429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:45.951980 15429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:45.957322 15429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:45.957736 15429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:45.969750 15429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:45.986048 15429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:45.986250 15429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260621 07:00:45.986483 15429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:45.986620 15429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:46.039995 15429 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:39625
I20260621 07:00:46.040022 15561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:39625 every 8 connection(s)
I20260621 07:00:46.041711 15429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:00:46.050678 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15429
I20260621 07:00:46.050871 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:00:46.051635 3031 external_mini_cluster.cc:1428] Setting key 05bc7ca6b1dc1d0e7a5e37a93365c734
Jun 21 07:00:46 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 07:00:46.116076 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:46.046641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.193:53655)
0621 07:00:46.081927 (+ 35286us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:46.081985 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:46.084524 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:46.091221 (+ 6697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:46.091235 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:46.091907 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:46.092744 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:46.092766 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:46.096616 (+ 3850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:46.096623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:46.098719 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:46.098735 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:46.099838 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:46.099850 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:46.100138 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:46.100962 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:46.100999 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:46.106529 (+ 5530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:46.110870 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:46.110880 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:46.110899 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:46.111545 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:46.111981 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:46.111985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:46.111991 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:46.112227 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:46.113124 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:46.113131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:46.113623 (+ 492us) client_negotiation.cc:769] Sending connection context
0621 07:00:46.113926 (+ 303us) client_negotiation.cc:240] Negotiation successful
0621 07:00:46.114543 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34934,"thread_start_us":43310,"threads_started":1}
I20260621 07:00:46.118129 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:46.047249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:53655 (local address 127.2.245.254:35507)
0621 07:00:46.080995 (+ 33746us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:46.081001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:46.085060 (+ 4059us) server_negotiation.cc:407] Connection header received
0621 07:00:46.085128 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:46.085134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:46.085238 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:46.085345 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:46.095260 (+ 9915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:46.096408 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:46.098873 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:46.099555 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:46.099681 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:46.106867 (+ 7186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:46.106877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:46.106884 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:46.106965 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:46.110702 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:46.111706 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:46.111718 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:46.111725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:46.111858 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:46.112349 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:46.112354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:46.112357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:46.112730 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:46.112890 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:46.117380 (+ 4490us) server_negotiation.cc:299] Negotiation successful
0621 07:00:46.117637 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33600,"thread_start_us":33943,"threads_started":1}
I20260621 07:00:46.119831 15562 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:46.120198 15562 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:46.121068 15562 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:46.132565 15349 ts_manager.cc:194] Registered new tserver with Master: 61444532d00e479fa63cfc88f617a5ca (127.2.245.193:39625)
I20260621 07:00:46.135450 15349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:53655
Jun 21 07:00:46 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
W20260621 07:00:46.176232 3031 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.2.245.194: real 0.122s user 0.003s sys 0.015s
I20260621 07:00:46.177322 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:00:46 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
W20260621 07:00:46.626824 15568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:46.627403 15568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:46.627635 15568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:46.639298 15568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:46.639514 15568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:46.639744 15568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:00:46.655225 15568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:46.657246 15568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:46.659068 15568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:46.682047 15575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:46.683095 15576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:46.700047 15578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:46.701205 15568 server_base.cc:1034] running on GCE node
I20260621 07:00:46.702075 15568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:46.712896 15568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:46.714385 15568 hybrid_clock.cc:648] HybridClock initialized: now 1782025246714273 us; error 114 us; skew 500 ppm
I20260621 07:00:46.714881 15568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:46 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025246, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:46.719340 15568 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:00:46.740077 15568 webserver.cc:460] Webserver started at http://127.2.245.194:36053/ using document root <none> and password file <none>
I20260621 07:00:46.741253 15568 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:46.741489 15568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:46.741966 15568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:00:46 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025246, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:46.765172 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "e6bd2f4fe5a44b908b107c80c32d0d64"
format_stamp: "Formatted at 2026-06-21 07:00:46 on dist-test-slave-9zdv"
server_key: "13c3a88bc3dabb66c609497aa85f4d71"
server_key_iv: "ad0e8189e2ec3f77be2386eade64fdbb"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:46.766430 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "e6bd2f4fe5a44b908b107c80c32d0d64"
format_stamp: "Formatted at 2026-06-21 07:00:46 on dist-test-slave-9zdv"
server_key: "13c3a88bc3dabb66c609497aa85f4d71"
server_key_iv: "ad0e8189e2ec3f77be2386eade64fdbb"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:46.780058 15568 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.003s sys 0.008s
I20260621 07:00:46.837553 15585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:46.839692 15568 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.000s
I20260621 07:00:46.839986 15568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "e6bd2f4fe5a44b908b107c80c32d0d64"
format_stamp: "Formatted at 2026-06-21 07:00:46 on dist-test-slave-9zdv"
server_key: "13c3a88bc3dabb66c609497aa85f4d71"
server_key_iv: "ad0e8189e2ec3f77be2386eade64fdbb"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:46.840320 15568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:46.880584 15568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:46.885836 15568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:46.886247 15568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:46.893584 15568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:46.901846 15568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:46.902033 15568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260621 07:00:46.902211 15568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:46.902315 15568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:46.998687 15568 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:36751
I20260621 07:00:46.998706 15704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:36751 every 8 connection(s)
I20260621 07:00:47.000144 15568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:00:47.002213 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15568
I20260621 07:00:47.002426 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:00:47.003127 3031 external_mini_cluster.cc:1428] Setting key 39e982a1e9f0914cec2363508275675b
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025246, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:47.078157 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.018682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:44479 (local address 127.2.245.254:35507)
0621 07:00:47.041140 (+ 22458us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:47.041147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:47.042998 (+ 1851us) server_negotiation.cc:407] Connection header received
0621 07:00:47.051126 (+ 8128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:47.051136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:47.051237 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:47.051326 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:47.053088 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.054321 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.061550 (+ 7229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.062311 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.062458 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.069558 (+ 7100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:47.069568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:47.069576 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:47.069639 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:47.073522 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.074597 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.074606 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.074612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.074763 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.075715 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.075723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.075729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.076147 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:47.076281 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:47.077372 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0621 07:00:47.077649 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22290,"thread_start_us":22493,"threads_started":1}
I20260621 07:00:47.080832 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.017928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.194:44479)
0621 07:00:47.041844 (+ 23916us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:47.041901 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:47.048992 (+ 7091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:47.051848 (+ 2856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:47.051856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:47.052327 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:47.052904 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:47.052923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.058730 (+ 5807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.058739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:47.061336 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:47.061356 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.062828 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.062836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:47.063204 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.064100 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:47.064137 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:47.069250 (+ 5113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:47.073750 (+ 4500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:47.073762 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:47.073785 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:47.074447 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:47.075292 (+ 845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:47.075299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:47.075305 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:47.075583 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:47.076374 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:47.076382 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:47.076905 (+ 523us) client_negotiation.cc:769] Sending connection context
0621 07:00:47.078668 (+ 1763us) client_negotiation.cc:240] Negotiation successful
0621 07:00:47.079327 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23570,"thread_start_us":31351,"threads_started":1}
I20260621 07:00:47.082590 15705 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:47.082930 15705 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:47.083742 15705 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:47.085371 15349 ts_manager.cc:194] Registered new tserver with Master: e6bd2f4fe5a44b908b107c80c32d0d64 (127.2.245.194:36751)
I20260621 07:00:47.086597 15349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:44479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:00:47.102234 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:00:47.139835 15562 heartbeater.cc:503] Master 127.2.245.254:35507 was elected leader, sending a full tablet report...
W20260621 07:00:47.430105 15711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:47.430364 15711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:47.430460 15711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:47.437668 15711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:47.437767 15711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:47.437927 15711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:00:47.447639 15711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:47.448839 15711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:47.450004 15711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:47.463042 15719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:47.463070 15717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:47.463407 15716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:47.464207 15711 server_base.cc:1034] running on GCE node
I20260621 07:00:47.464787 15711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:47.465747 15711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:47.466995 15711 hybrid_clock.cc:648] HybridClock initialized: now 1782025247466913 us; error 30 us; skew 500 ppm
I20260621 07:00:47.467332 15711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:47.470392 15711 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:00:47.471942 15711 webserver.cc:460] Webserver started at http://127.2.245.195:41523/ using document root <none> and password file <none>
I20260621 07:00:47.472704 15711 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:47.472851 15711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:47.473152 15711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:47.487624 15711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "4f51a9cc3fd142859784dbb4e794c396"
format_stamp: "Formatted at 2026-06-21 07:00:47 on dist-test-slave-9zdv"
server_key: "4361bf3c1739202727292495017d9bae"
server_key_iv: "bec5a942e5d43c021d855ac48793607d"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:47.488569 15711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "4f51a9cc3fd142859784dbb4e794c396"
format_stamp: "Formatted at 2026-06-21 07:00:47 on dist-test-slave-9zdv"
server_key: "4361bf3c1739202727292495017d9bae"
server_key_iv: "bec5a942e5d43c021d855ac48793607d"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:47.494180 15711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:00:47.510164 15726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:47.512080 15711 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260621 07:00:47.512262 15711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "4f51a9cc3fd142859784dbb4e794c396"
format_stamp: "Formatted at 2026-06-21 07:00:47 on dist-test-slave-9zdv"
server_key: "4361bf3c1739202727292495017d9bae"
server_key_iv: "bec5a942e5d43c021d855ac48793607d"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:47.512508 15711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:47.551268 15711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:47.555123 15711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:47.555377 15711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:47.556313 15711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:47.557600 15711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:47.557670 15711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:47.557750 15711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:47.557791 15711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:47.591382 15711 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:32907
I20260621 07:00:47.591410 15838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:32907 every 8 connection(s)
I20260621 07:00:47.592783 15711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:00:47.594864 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15711
I20260621 07:00:47.595098 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:00:47.595642 3031 external_mini_cluster.cc:1428] Setting key 694b95163d130a0d0d030ebf2b57b184
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:47.613341 15841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.595566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:44151 (local address 127.2.245.254:35507)
0621 07:00:47.596172 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:47.596178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:47.597224 (+ 1046us) server_negotiation.cc:407] Connection header received
0621 07:00:47.599479 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:47.599483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:47.599545 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:47.599631 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:47.601107 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.601968 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.603518 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.603960 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.604059 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.607452 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:47.607457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:47.607460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:47.607491 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:47.610182 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.611111 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.611123 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.611126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.611251 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.611778 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.611783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.611785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.612096 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:47.612212 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:47.612935 (+ 723us) server_negotiation.cc:299] Negotiation successful
0621 07:00:47.613101 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":285,"threads_started":1}
I20260621 07:00:47.614923 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.596214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.195:44151)
0621 07:00:47.597053 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:47.597122 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:47.599184 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:47.599849 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:47.599865 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:47.600276 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:47.600931 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:47.600952 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.602104 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.602108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:47.603385 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:47.603397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.604148 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.604152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:47.604341 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.605058 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:47.605084 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:47.607233 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:47.610327 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:47.610337 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:47.610355 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:47.610989 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:47.611363 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:47.611385 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:47.611388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:47.611646 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:47.612206 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:47.612211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:47.612549 (+ 338us) client_negotiation.cc:769] Sending connection context
0621 07:00:47.613558 (+ 1009us) client_negotiation.cc:240] Negotiation successful
0621 07:00:47.613934 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":326,"threads_started":1}
I20260621 07:00:47.616268 15839 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:47.616568 15839 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:47.617302 15839 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:47.618687 15349 ts_manager.cc:194] Registered new tserver with Master: 4f51a9cc3fd142859784dbb4e794c396 (127.2.245.195:32907)
I20260621 07:00:47.619568 15349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:44151
I20260621 07:00:47.623970 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:00:47.625623 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15429
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:00:47.642889 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.626570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.2.245.193:39625)
0621 07:00:47.627252 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:47.627261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:47.627287 (+ 26us) server_negotiation.cc:407] Connection header received
0621 07:00:47.627364 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:47.627369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:47.627549 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:47.627713 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:47.629257 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.630250 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.631817 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.632329 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.632497 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.636811 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:47.636823 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:47.636837 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:47.636872 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:47.640003 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.640707 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.640714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.640716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.640858 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.641296 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.641300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.641302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.641614 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:47.641936 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:47.642251 (+ 315us) server_negotiation.cc:299] Negotiation successful
0621 07:00:47.642536 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":304,"threads_started":1}
I20260621 07:00:47.644196 15557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:47.728655 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15429
I20260621 07:00:47.744401 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15568
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:00:47.761695 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.745339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.2.245.194:36751)
0621 07:00:47.745939 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:47.745945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:47.745972 (+ 27us) server_negotiation.cc:407] Connection header received
0621 07:00:47.746033 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:47.746039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:47.746264 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:47.746465 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:47.747763 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.748856 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.750322 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.750811 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.750909 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.756013 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:47.756023 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:47.756037 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:47.756076 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:47.758786 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.759839 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.759847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.759849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.759993 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.760337 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.760340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.760342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.760664 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:47.760864 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:47.761191 (+ 327us) server_negotiation.cc:299] Negotiation successful
0621 07:00:47.761422 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":325,"threads_started":1}
I20260621 07:00:47.763028 15700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:47.846328 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15568
I20260621 07:00:47.862036 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15711
Jun 21 07:00:47 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:00:47.877723 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:47.863024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54350 (local address 127.2.245.195:32907)
0621 07:00:47.863613 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:47.863620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:47.863648 (+ 28us) server_negotiation.cc:407] Connection header received
0621 07:00:47.863709 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:47.863715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:47.863924 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:47.864050 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:47.865323 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.866337 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.867643 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:47.868212 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:47.868308 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:47.872702 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:47.872711 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:47.872725 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:47.872755 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:47.875388 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.876043 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.876049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.876051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.876163 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:47.876494 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:47.876496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:47.876498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:47.876791 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:47.877008 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:47.877269 (+ 261us) server_negotiation.cc:299] Negotiation successful
0621 07:00:47.877461 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":266,"threads_started":1}
I20260621 07:00:47.879082 15834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:47.960572 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15711
I20260621 07:00:47.976372 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15317
I20260621 07:00:47.977617 15379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:48.082378 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15317
I20260621 07:00:48.106491 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35507
--webserver_interface=127.2.245.254
--webserver_port=39011
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35507
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:00:48.428311 15851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:48.428620 15851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:48.428761 15851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:48.435937 15851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:48.436040 15851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:48.436151 15851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:48.436252 15851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:48.436336 15851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:48.446314 15851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35507
--ranger_config_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=39011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:48.447671 15851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:48.448972 15851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:48.460645 15856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:48.460695 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:48.460729 15859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:48.461534 15851 server_base.cc:1034] running on GCE node
I20260621 07:00:48.462248 15851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:48.463713 15851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:48.464922 15851 hybrid_clock.cc:648] HybridClock initialized: now 1782025248464891 us; error 35 us; skew 500 ppm
I20260621 07:00:48.465282 15851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:48 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025248, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:48.468493 15851 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:00:48.469959 15851 webserver.cc:460] Webserver started at http://127.2.245.254:39011/ using document root <none> and password file <none>
I20260621 07:00:48.470624 15851 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:48.470770 15851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:00:48 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025248, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:48.517105 15851 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260621 07:00:48.520293 15867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:48.521755 15851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:00:48.521974 15851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "7bb4e8bd315c453c960c499648f5322d"
format_stamp: "Formatted at 2026-06-21 07:00:44 on dist-test-slave-9zdv"
server_key: "772323d1bcb121358484875ff87001e3"
server_key_iv: "3b7d13bd0d0ce2137f8a723061d40e4f"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:48.523043 15851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:48.542320 15851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:48.546231 15851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:48.546541 15851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:48.564568 15851 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35507
I20260621 07:00:48.564589 15918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35507 every 8 connection(s)
I20260621 07:00:48.566026 15851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:00:48.569093 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15851
I20260621 07:00:48.569897 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:39625
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:35507
--webserver_port=39361
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:00:48.573402 15919 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:00:48.577736 15919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Bootstrap starting.
I20260621 07:00:48.596287 15919 log.cc:824] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:48.605470 15919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:00:48.605988 15919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Bootstrap complete.
I20260621 07:00:48.614722 15919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:48.615717 15919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb4e8bd315c453c960c499648f5322d, State: Initialized, Role: FOLLOWER
I20260621 07:00:48.616325 15919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:48.616550 15919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:00:48.616664 15919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:00:48.616871 15919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:00:48.620677 15919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:48.621196 15919 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bb4e8bd315c453c960c499648f5322d; no voters:
I20260621 07:00:48.621848 15919 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:00:48.621987 15923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:00:48.623338 15923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [term 2 LEADER]: Becoming Leader. State: Replica: 7bb4e8bd315c453c960c499648f5322d, State: Running, Role: LEADER
I20260621 07:00:48.623854 15923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } }
I20260621 07:00:48.624181 15919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:48.626216 15925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bb4e8bd315c453c960c499648f5322d. Latest consensus state: current_term: 2 leader_uuid: "7bb4e8bd315c453c960c499648f5322d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } } }
I20260621 07:00:48.626606 15925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:48.627238 15924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7bb4e8bd315c453c960c499648f5322d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb4e8bd315c453c960c499648f5322d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35507 } } }
I20260621 07:00:48.627542 15924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:48.632145 15929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:00:48.638983 15929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:00:48.640760 15929 catalog_manager.cc:1259] Loaded cluster ID: 82fa33bd116e44b4b2d7df8073758d80
I20260621 07:00:48.640887 15929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:00:48.646298 15929 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:00:48.648178 15929 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Loaded TSK: 0
I20260621 07:00:48.649997 15929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:00:48.656137 15919 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260621 07:00:49.010514 15921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:49.011035 15921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:49.011246 15921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:49.023150 15921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:49.023360 15921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:49.023578 15921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:00:49.039510 15921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:39625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=39361
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:49.041297 15921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:49.043084 15921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:49.099030 15957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:49.102538 15960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:49.103081 15956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:49.103703 15921 server_base.cc:1034] running on GCE node
I20260621 07:00:49.104499 15921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:49.110080 15921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:49.112807 15921 hybrid_clock.cc:648] HybridClock initialized: now 1782025249112729 us; error 70 us; skew 500 ppm
I20260621 07:00:49.113274 15921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:49.118582 15921 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:00:49.134625 15921 webserver.cc:460] Webserver started at http://127.2.245.193:39361/ using document root <none> and password file <none>
I20260621 07:00:49.135461 15921 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:49.135625 15921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:49.183241 15921 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.003s sys 0.005s
I20260621 07:00:49.201571 15968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:49.205504 15921 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.013s sys 0.005s
I20260621 07:00:49.205785 15921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "61444532d00e479fa63cfc88f617a5ca"
format_stamp: "Formatted at 2026-06-21 07:00:45 on dist-test-slave-9zdv"
server_key: "2f96568c9bf6372450741d83194fed1e"
server_key_iv: "b393c502709fededc164341bcf818678"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:49.207304 15921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:49.242692 15921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:49.247176 15921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:49.247474 15921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:49.248472 15921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:49.250028 15921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:49.250139 15921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:49.250247 15921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:49.250319 15921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:49.296618 15921 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:39625
I20260621 07:00:49.296656 16080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:39625 every 8 connection(s)
I20260621 07:00:49.298460 15921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:00:49.304634 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 15921
I20260621 07:00:49.305502 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:36751
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:35507
--webserver_port=36053
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:49.330148 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:49.301899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.193:54123)
0621 07:00:49.302709 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:49.302761 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:49.305331 (+ 2570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:49.307254 (+ 1923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:49.307265 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:49.307900 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:49.308709 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:49.308738 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:49.311174 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:49.311180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:49.313641 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:49.313669 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:49.314768 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:49.314776 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:49.315091 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:49.315856 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:49.315885 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:49.319113 (+ 3228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:49.324099 (+ 4986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:49.324111 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:49.324130 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:49.324811 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:49.325404 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:49.325412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:49.325418 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:49.325653 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:49.326989 (+ 1336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:49.327014 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:49.327524 (+ 510us) client_negotiation.cc:769] Sending connection context
0621 07:00:49.327870 (+ 346us) client_negotiation.cc:240] Negotiation successful
0621 07:00:49.328523 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":361,"threads_started":1}
I20260621 07:00:49.333036 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:49.301939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:54123 (local address 127.2.245.254:35507)
0621 07:00:49.303690 (+ 1751us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:49.303708 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:49.303762 (+ 54us) server_negotiation.cc:407] Connection header received
0621 07:00:49.306126 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:49.306147 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:49.306656 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:49.307064 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:49.308964 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:49.311003 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:49.313816 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:49.314527 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:49.314665 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:49.319463 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:49.319475 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:49.319498 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:49.319557 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:49.323911 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:49.325000 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:49.325010 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:49.325019 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:49.325232 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:49.325806 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:49.325813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:49.325819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:49.326522 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:49.326744 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:49.331015 (+ 4271us) server_negotiation.cc:299] Negotiation successful
0621 07:00:49.331549 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1501,"thread_start_us":1274,"threads_started":1}
I20260621 07:00:49.337128 16081 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:49.337426 16081 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:49.338208 16081 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
W20260621 07:00:49.341262 16081 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:35507 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:00:49.341463 16081 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:49.341714 16081 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:49.342900 16081 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:49.343163 16081 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
W20260621 07:00:49.344269 16081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:00:49.735242 16085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:49.735642 16085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:49.735838 16085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:49.747349 16085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:49.747542 16085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:49.747749 16085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:00:49.763248 16085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:36751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=36053
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:49.764842 16085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:49.766553 16085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:49.782703 16101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:49.784523 16102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:49.790987 16104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:49.792958 16085 server_base.cc:1034] running on GCE node
I20260621 07:00:49.793768 16085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:49.795186 16085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:49.802799 16085 hybrid_clock.cc:648] HybridClock initialized: now 1782025249802714 us; error 65 us; skew 500 ppm
I20260621 07:00:49.803253 16085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:49.807648 16085 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:00:49.837703 16085 webserver.cc:460] Webserver started at http://127.2.245.194:36053/ using document root <none> and password file <none>
I20260621 07:00:49.839210 16085 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:49.839471 16085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:00:49 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:49.879779 16085 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.004s
I20260621 07:00:49.884104 16113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:49.886185 16085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:00:49.886514 16085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "e6bd2f4fe5a44b908b107c80c32d0d64"
format_stamp: "Formatted at 2026-06-21 07:00:46 on dist-test-slave-9zdv"
server_key: "13c3a88bc3dabb66c609497aa85f4d71"
server_key_iv: "ad0e8189e2ec3f77be2386eade64fdbb"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:49.888002 16085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:49.945084 16085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:49.950392 16085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:49.950713 16085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:49.952000 16085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:00:49.958029 16085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:49.958155 16085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:49.958319 16085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:49.958415 16085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:50.025885 16085 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:36751
I20260621 07:00:50.025965 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:36751 every 8 connection(s)
I20260621 07:00:50.028000 16085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:00:50.031401 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16085
I20260621 07:00:50.032166 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.245.212:34967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:32907
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:35507
--webserver_port=41523
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:50.077121 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.030808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:34951 (local address 127.2.245.254:35507)
0621 07:00:50.048720 (+ 17912us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.048728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.049962 (+ 1234us) server_negotiation.cc:407] Connection header received
0621 07:00:50.052476 (+ 2514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.052483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.052585 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.052681 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:50.055142 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.056288 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.058828 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.059532 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.059641 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.066501 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:50.066510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:50.066514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:50.066559 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:50.072941 (+ 6382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.074032 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.074040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.074044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.074211 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.074783 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.074789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.074793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.075222 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:50.075389 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.076473 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.076706 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17748,"thread_start_us":19141,"threads_started":1}
I20260621 07:00:50.079936 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.038555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.194:34951)
0621 07:00:50.049354 (+ 10799us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:50.049427 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:50.052097 (+ 2670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:50.053321 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:50.053334 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:50.054025 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:50.054903 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.054932 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.056524 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.056530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.058657 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.058672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.060037 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.060045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.060347 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.061219 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:50.061260 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:50.066152 (+ 4892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:50.073146 (+ 6994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:50.073158 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:50.073177 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:50.073867 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:50.074386 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:50.074392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:50.074396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:50.074637 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:50.075488 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:50.075497 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:50.075965 (+ 468us) client_negotiation.cc:769] Sending connection context
0621 07:00:50.077760 (+ 1795us) client_negotiation.cc:240] Negotiation successful
0621 07:00:50.078387 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10456,"thread_start_us":10231,"threads_started":1}
I20260621 07:00:50.082067 16226 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:50.082428 16226 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:50.083372 16226 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:50.086026 15884 ts_manager.cc:194] Registered new tserver with Master: e6bd2f4fe5a44b908b107c80c32d0d64 (127.2.245.194:36751)
I20260621 07:00:50.088204 15884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:34951
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:50.344444 16081 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:50.344872 16081 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:50.346153 15884 ts_manager.cc:194] Registered new tserver with Master: 61444532d00e479fa63cfc88f617a5ca (127.2.245.193:39625)
I20260621 07:00:50.347324 15884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:54123
W20260621 07:00:50.458251 16230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:50.458696 16230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:50.459450 16230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:50.470851 16230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:50.471016 16230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:50.471254 16230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:00:50.486461 16230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.245.212:34967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:32907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=41523
--tserver_master_addrs=127.2.245.254:35507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:50.488088 16230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:50.489703 16230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:50.507584 16235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:50.508162 16236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:50.515801 16230 server_base.cc:1034] running on GCE node
W20260621 07:00:50.516702 16238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:50.517539 16230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:50.518931 16230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:50.520205 16230 hybrid_clock.cc:648] HybridClock initialized: now 1782025250520126 us; error 65 us; skew 500 ppm
I20260621 07:00:50.520586 16230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025250, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:50.524677 16230 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:00:50.532349 16230 webserver.cc:460] Webserver started at http://127.2.245.195:41523/ using document root <none> and password file <none>
I20260621 07:00:50.533488 16230 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:50.533663 16230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025250, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:50.559538 16230 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:00:50.575623 16246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:50.577574 16230 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.009s sys 0.007s
I20260621 07:00:50.577814 16230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "4f51a9cc3fd142859784dbb4e794c396"
format_stamp: "Formatted at 2026-06-21 07:00:47 on dist-test-slave-9zdv"
server_key: "4361bf3c1739202727292495017d9bae"
server_key_iv: "bec5a942e5d43c021d855ac48793607d"
server_key_version: "kuduclusterkey@0"
I20260621 07:00:50.578955 16230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:50.614543 16230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:50.619521 16230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:50.619827 16230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:50.621237 16230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025249, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for HTTP/127.2.245.212@KRBTEST.COM
I20260621 07:00:50.623994 16230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:00:50.624145 16230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:50.624286 16230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:00:50.624400 16230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:50.748128 16230 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:32907
I20260621 07:00:50.748174 16359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:32907 every 8 connection(s)
I20260621 07:00:50.749615 16230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:00:50.753496 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16230
Jun 21 07:00:50 dist-test-slave-9zdv krb5kdc[14873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025250, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:50.808844 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.766471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:35507 (local address 127.2.245.195:37891)
0621 07:00:50.780848 (+ 14377us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:50.780926 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:50.787366 (+ 6440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:50.788580 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:50.788593 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:50.789253 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:50.790107 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.790135 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.791694 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.791700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.793704 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.793730 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.795070 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.795077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.795412 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.796231 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:50.796266 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:50.798803 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:50.803238 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:50.803248 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:50.803267 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:50.803903 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:50.804356 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:50.804361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:50.804364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:50.804645 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:50.805389 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:50.805409 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:50.805873 (+ 464us) client_negotiation.cc:769] Sending connection context
0621 07:00:50.806188 (+ 315us) client_negotiation.cc:240] Negotiation successful
0621 07:00:50.807094 (+ 906us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13966,"thread_start_us":16302,"threads_started":1}
I20260621 07:00:50.809445 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.767263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:37891 (local address 127.2.245.254:35507)
0621 07:00:50.786155 (+ 18892us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.786159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.786180 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:00:50.787820 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.787825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.787919 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.788017 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:50.790320 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.791504 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.793898 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.794606 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.794739 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.799084 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:50.799091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:50.799096 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:50.799142 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:50.803033 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.804065 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.804072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.804078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.804214 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.804800 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.804806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.804811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.805157 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:50.805306 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.807377 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.809096 (+ 1719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18796,"thread_start_us":18650,"threads_started":1}
I20260621 07:00:50.810987 16360 heartbeater.cc:347] Connected to a master server at 127.2.245.254:35507
I20260621 07:00:50.811365 16360 heartbeater.cc:464] Registering TS with master...
I20260621 07:00:50.812318 16360 heartbeater.cc:511] Master 127.2.245.254:35507 requested a full tablet report, sending...
I20260621 07:00:50.814294 15884 ts_manager.cc:194] Registered new tserver with Master: 4f51a9cc3fd142859784dbb4e794c396 (127.2.245.195:32907)
I20260621 07:00:50.815341 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.754744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.2.245.254:35507)
0621 07:00:50.784504 (+ 29760us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.784513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.786317 (+ 1804us) server_negotiation.cc:407] Connection header received
0621 07:00:50.786629 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.786635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.786728 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.786829 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:50.795422 (+ 8593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.796724 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.799387 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.800193 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.800299 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.805657 (+ 5358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:50.805663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:50.805668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:50.805705 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:50.808906 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.810915 (+ 2009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.810920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.810923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.811106 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.813435 (+ 2329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.813440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.813444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.813775 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:50.814042 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.814661 (+ 619us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.814906 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29614,"thread_start_us":29450,"threads_started":1}
I20260621 07:00:50.815533 15884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:37891
I20260621 07:00:50.819062 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:00:50.862486 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.825283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.2.245.254:35507)
0621 07:00:50.825504 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.825511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.845301 (+ 19790us) server_negotiation.cc:407] Connection header received
0621 07:00:50.845519 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.845525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.845627 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.845739 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:50.848017 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.849120 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.851244 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.851882 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.851990 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.855596 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:50.855601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:50.855606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:50.855648 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:50.859166 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.860066 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.860071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.860074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.860232 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:50.860731 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:50.860737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:50.860740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:50.861090 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:50.861314 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.861885 (+ 571us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.862120 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:00:50.869690 15884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:00:50.882120 15884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:00:50.936264 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.923962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:32907 (local address 127.0.0.1:54362)
0621 07:00:50.925871 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:50.925894 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:50.926033 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:50.927233 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:50.927243 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:50.927302 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:50.927908 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.927922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.930107 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.930113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.932653 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.932671 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.935129 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.935136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.935336 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.935391 (+ 55us) client_negotiation.cc:769] Sending connection context
0621 07:00:50.935569 (+ 178us) client_negotiation.cc:240] Negotiation successful
0621 07:00:50.935790 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1747,"mutex_wait_us":28,"spinlock_wait_cycles":17408,"thread_start_us":1072,"threads_started":1}
I20260621 07:00:50.937242 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.924333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.2.245.195:32907)
0621 07:00:50.925671 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.925680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.926161 (+ 481us) server_negotiation.cc:407] Connection header received
0621 07:00:50.926276 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.926284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.926624 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.926832 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:50.928127 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.929935 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.932854 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.934500 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.934652 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.934955 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.936695 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.936883 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1132,"thread_start_us":917,"threads_started":1}
I20260621 07:00:50.941890 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.907280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36751 (local address 127.0.0.1:37180)
0621 07:00:50.924883 (+ 17603us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:50.924905 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:50.925030 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:50.927236 (+ 2206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:50.927244 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:50.927297 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:50.927842 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.927859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.929819 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.929826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.935564 (+ 5738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.935578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.941002 (+ 5424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.941008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.941211 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.941249 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 07:00:50.941335 (+ 86us) client_negotiation.cc:240] Negotiation successful
0621 07:00:50.941478 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17497,"mutex_wait_us":3292,"spinlock_wait_cycles":1792,"thread_start_us":15675,"threads_started":1}
I20260621 07:00:50.942817 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.907350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.2.245.194:36751)
0621 07:00:50.926196 (+ 18846us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.926204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.926232 (+ 28us) server_negotiation.cc:407] Connection header received
0621 07:00:50.926327 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.926335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.926619 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.926951 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:50.928075 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.929645 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.937508 (+ 7863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.939097 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.939212 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.939474 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.942266 (+ 2792us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.942468 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18693,"thread_start_us":18564,"threads_started":1}
I20260621 07:00:50.942374 16295 tablet_service.cc:1467] Processing CreateTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 (DEFAULT_TABLE table=test-table [id=786c6b7fccca462c9df8e2d844f95203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:50.944252 16295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d419e18fd6e4bcc8edca21a3a842598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:50.944310 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.906836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38000)
0621 07:00:50.924593 (+ 17757us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:50.924656 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:50.924854 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:50.928654 (+ 3800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:50.928660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:50.928687 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:50.929040 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.929052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.931120 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.931126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.934653 (+ 3527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:50.934668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.943410 (+ 8742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.943417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:50.943623 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.943660 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 07:00:50.943738 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 07:00:50.943926 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16080,"mutex_wait_us":3,"thread_start_us":18706,"threads_started":1}
I20260621 07:00:50.945938 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:50.907488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.2.245.193:39625)
0621 07:00:50.927768 (+ 20280us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:50.927778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:50.927811 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:00:50.927925 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:50.927934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:50.928204 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:50.928406 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:50.929270 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.930926 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.934864 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:50.937293 (+ 2429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:50.945022 (+ 7729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:50.945301 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:50.945382 (+ 81us) server_negotiation.cc:299] Negotiation successful
0621 07:00:50.945573 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20082,"thread_start_us":17975,"threads_started":1}
I20260621 07:00:50.948230 16161 tablet_service.cc:1467] Processing CreateTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 (DEFAULT_TABLE table=test-table [id=786c6b7fccca462c9df8e2d844f95203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:50.949848 16161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d419e18fd6e4bcc8edca21a3a842598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:50.949658 16016 tablet_service.cc:1467] Processing CreateTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 (DEFAULT_TABLE table=test-table [id=786c6b7fccca462c9df8e2d844f95203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:00:50.951149 16016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d419e18fd6e4bcc8edca21a3a842598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:50.981201 16384 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Bootstrap starting.
I20260621 07:00:50.981385 16385 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Bootstrap starting.
I20260621 07:00:50.983536 16383 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Bootstrap starting.
I20260621 07:00:50.986923 16384 tablet_bootstrap.cc:654] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:50.988145 16384 log.cc:824] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:50.988596 16385 tablet_bootstrap.cc:654] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:50.990296 16383 tablet_bootstrap.cc:654] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:50.991739 16383 log.cc:824] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:50.993031 16385 log.cc:824] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:50.998872 16383 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: No bootstrap required, opened a new log
I20260621 07:00:50.999191 16383 ts_tablet_manager.cc:1397] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.006s
I20260621 07:00:50.999418 16385 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: No bootstrap required, opened a new log
I20260621 07:00:50.999428 16384 tablet_bootstrap.cc:492] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: No bootstrap required, opened a new log
I20260621 07:00:50.999696 16384 ts_tablet_manager.cc:1397] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20260621 07:00:50.999696 16385 ts_tablet_manager.cc:1397] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.002s
I20260621 07:00:51.010191 16383 raft_consensus.cc:348] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.010828 16383 raft_consensus.cc:374] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:51.011006 16383 raft_consensus.cc:729] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61444532d00e479fa63cfc88f617a5ca, State: Initialized, Role: FOLLOWER
I20260621 07:00:51.011412 16385 raft_consensus.cc:348] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.011930 16385 raft_consensus.cc:374] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:51.012079 16385 raft_consensus.cc:729] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bd2f4fe5a44b908b107c80c32d0d64, State: Initialized, Role: FOLLOWER
I20260621 07:00:51.011893 16383 consensus_queue.cc:260] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.012861 16385 consensus_queue.cc:260] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.013149 16384 raft_consensus.cc:348] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.013734 16384 raft_consensus.cc:374] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:51.013902 16384 raft_consensus.cc:729] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51a9cc3fd142859784dbb4e794c396, State: Initialized, Role: FOLLOWER
I20260621 07:00:51.014734 16384 consensus_queue.cc:260] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.017457 16383 ts_tablet_manager.cc:1428] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260621 07:00:51.017850 16081 heartbeater.cc:503] Master 127.2.245.254:35507 was elected leader, sending a full tablet report...
I20260621 07:00:51.025177 16226 heartbeater.cc:503] Master 127.2.245.254:35507 was elected leader, sending a full tablet report...
I20260621 07:00:51.026031 16385 ts_tablet_manager.cc:1428] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Time spent starting tablet: real 0.026s user 0.014s sys 0.012s
I20260621 07:00:51.030575 16360 heartbeater.cc:503] Master 127.2.245.254:35507 was elected leader, sending a full tablet report...
I20260621 07:00:51.031360 16384 ts_tablet_manager.cc:1428] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Time spent starting tablet: real 0.032s user 0.013s sys 0.014s
W20260621 07:00:51.036075 16363 tablet.cc:2369] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:00:51.053138 16227 tablet.cc:2369] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:00:51.053195 16082 tablet.cc:2369] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:00:51.172358 16396 raft_consensus.cc:482] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:00:51.172700 16396 raft_consensus.cc:504] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.174383 16396 leader_election.cc:288] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61444532d00e479fa63cfc88f617a5ca (127.2.245.193:39625), e6bd2f4fe5a44b908b107c80c32d0d64 (127.2.245.194:36751)
I20260621 07:00:51.181401 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.174639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.2.245.195:46473)
0621 07:00:51.174930 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.175455 (+ 525us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:51.175601 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:51.176015 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:51.176023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:51.176057 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:51.176251 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:51.176262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.177681 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.177689 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:51.179425 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:51.179438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.180696 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.180701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:51.180874 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.180919 (+ 45us) client_negotiation.cc:769] Sending connection context
0621 07:00:51.181019 (+ 100us) client_negotiation.cc:240] Negotiation successful
0621 07:00:51.181150 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260621 07:00:51.181468 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.175194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:46473 (local address 127.2.245.193:39625)
0621 07:00:51.175415 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.175420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.175714 (+ 294us) server_negotiation.cc:407] Connection header received
0621 07:00:51.175788 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.175794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.175876 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.175972 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:51.176409 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.177525 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.179591 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.180516 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.180699 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.180769 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.181064 (+ 295us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.181194 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260621 07:00:51.182463 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.175034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:49299 (local address 127.2.245.194:36751)
0621 07:00:51.175303 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.175311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.175844 (+ 533us) server_negotiation.cc:407] Connection header received
0621 07:00:51.176071 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.176076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.176161 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.176228 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:00:51.176881 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.177543 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.180222 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.181370 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.181539 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.181606 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.182091 (+ 485us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.182217 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260621 07:00:51.182530 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.174863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36751 (local address 127.2.245.195:49299)
0621 07:00:51.175731 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.175745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:51.175888 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:51.176410 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:51.176414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:51.176444 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:00:51.176712 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:51.176718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.177719 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.177724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:51.180045 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:51.180060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.181735 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.181740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:51.181852 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.181873 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:00:51.182035 (+ 162us) client_negotiation.cc:240] Negotiation successful
0621 07:00:51.182173 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":5,"thread_start_us":581,"threads_started":1}
I20260621 07:00:51.182912 16036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d419e18fd6e4bcc8edca21a3a842598" candidate_uuid: "4f51a9cc3fd142859784dbb4e794c396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61444532d00e479fa63cfc88f617a5ca" is_pre_election: true
I20260621 07:00:51.183604 16036 raft_consensus.cc:2454] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51a9cc3fd142859784dbb4e794c396 in term 0.
I20260621 07:00:51.183574 16181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d419e18fd6e4bcc8edca21a3a842598" candidate_uuid: "4f51a9cc3fd142859784dbb4e794c396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" is_pre_election: true
I20260621 07:00:51.184194 16181 raft_consensus.cc:2454] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51a9cc3fd142859784dbb4e794c396 in term 0.
I20260621 07:00:51.184378 16249 leader_election.cc:302] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51a9cc3fd142859784dbb4e794c396, 61444532d00e479fa63cfc88f617a5ca; no voters:
I20260621 07:00:51.184825 16396 raft_consensus.cc:2781] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:00:51.184967 16396 raft_consensus.cc:482] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:00:51.185094 16396 raft_consensus.cc:3037] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:51.187383 16396 raft_consensus.cc:504] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.188036 16396 leader_election.cc:288] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [CANDIDATE]: Term 1 election: Requested vote from peers 61444532d00e479fa63cfc88f617a5ca (127.2.245.193:39625), e6bd2f4fe5a44b908b107c80c32d0d64 (127.2.245.194:36751)
I20260621 07:00:51.188454 16036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d419e18fd6e4bcc8edca21a3a842598" candidate_uuid: "4f51a9cc3fd142859784dbb4e794c396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61444532d00e479fa63cfc88f617a5ca"
I20260621 07:00:51.188560 16181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d419e18fd6e4bcc8edca21a3a842598" candidate_uuid: "4f51a9cc3fd142859784dbb4e794c396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64"
I20260621 07:00:51.188669 16036 raft_consensus.cc:3037] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:51.188764 16181 raft_consensus.cc:3037] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:51.190846 16036 raft_consensus.cc:2454] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51a9cc3fd142859784dbb4e794c396 in term 1.
I20260621 07:00:51.191314 16249 leader_election.cc:302] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51a9cc3fd142859784dbb4e794c396, 61444532d00e479fa63cfc88f617a5ca; no voters:
I20260621 07:00:51.191622 16396 raft_consensus.cc:2781] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:00:51.191845 16181 raft_consensus.cc:2454] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51a9cc3fd142859784dbb4e794c396 in term 1.
I20260621 07:00:51.192188 16396 raft_consensus.cc:686] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 1 LEADER]: Becoming Leader. State: Replica: 4f51a9cc3fd142859784dbb4e794c396, State: Running, Role: LEADER
I20260621 07:00:51.193027 16396 consensus_queue.cc:237] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } }
I20260621 07:00:51.197956 15884 catalog_manager.cc:5495] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f51a9cc3fd142859784dbb4e794c396 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "4f51a9cc3fd142859784dbb4e794c396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51a9cc3fd142859784dbb4e794c396" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 32907 } health_report { overall_health: HEALTHY } } }
I20260621 07:00:51.217846 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.212363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.2.245.195:32907)
0621 07:00:51.212614 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.212621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.212759 (+ 138us) server_negotiation.cc:407] Connection header received
0621 07:00:51.212977 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.212980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.213062 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.213212 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:00:51.213584 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.214311 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.215738 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.216126 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.216263 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.216756 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:00:51.217255 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:00:51.217414 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.217477 (+ 63us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.217564 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:00:51.234874 16036 raft_consensus.cc:1261] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 1 FOLLOWER]: Refusing update from remote peer 4f51a9cc3fd142859784dbb4e794c396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:51.235989 16398 consensus_queue.cc:1035] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61444532d00e479fa63cfc88f617a5ca" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:00:51.236251 16181 raft_consensus.cc:1261] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 1 FOLLOWER]: Refusing update from remote peer 4f51a9cc3fd142859784dbb4e794c396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:51.237049 16398 consensus_queue.cc:1035] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6bd2f4fe5a44b908b107c80c32d0d64" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:00:51.272578 16403 mvcc.cc:205] Tried to move back new op lower bound from 7299175429047054336 to 7299175428891025408. Current Snapshot: MvccSnapshot[applied={T|T < 7299175429047054336}]
I20260621 07:00:51.290340 15884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:00:51.290614 15884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:00:51.294720 15884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7bb4e8bd315c453c960c499648f5322d: Sending DeleteTablet for 3 replicas of tablet 1d419e18fd6e4bcc8edca21a3a842598
I20260621 07:00:51.295979 16016 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:51 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:38000
I20260621 07:00:51.296162 16295 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:51 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:54362
I20260621 07:00:51.296201 16161 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d419e18fd6e4bcc8edca21a3a842598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:00:51 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37180
I20260621 07:00:51.305042 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15921
I20260621 07:00:51.325654 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.306685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.2.245.193:39625)
0621 07:00:51.306847 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.306852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.306869 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:00:51.306923 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.306929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.307028 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.307132 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:51.308988 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.310184 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.312908 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.313718 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.313823 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.317198 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:51.317213 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:51.317231 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:51.317268 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:51.321745 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.322601 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.322609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.322613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.322766 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.323221 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.323226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.323229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.323695 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:51.324017 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.324968 (+ 951us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.325245 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:00:51.327072 16076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:51.330173 16414 tablet_replica.cc:330] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: stopping tablet replica
I20260621 07:00:51.330865 16414 raft_consensus.cc:2229] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:00:51.331624 16414 raft_consensus.cc:2258] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:51.334013 16414 ts_tablet_manager.cc:1905] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:51.343416 16414 ts_tablet_manager.cc:1918] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:00:51.343802 16414 log.cc:1196] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/1d419e18fd6e4bcc8edca21a3a842598
I20260621 07:00:51.344542 16414 ts_tablet_manager.cc:1939] T 1d419e18fd6e4bcc8edca21a3a842598 P e6bd2f4fe5a44b908b107c80c32d0d64: Deleting consensus metadata
I20260621 07:00:51.346681 15869 catalog_manager.cc:4841] TS e6bd2f4fe5a44b908b107c80c32d0d64 (127.2.245.194:36751): tablet 1d419e18fd6e4bcc8edca21a3a842598 (table test-table [id=786c6b7fccca462c9df8e2d844f95203]) successfully deleted
I20260621 07:00:51.347373 16413 tablet_replica.cc:330] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: stopping tablet replica
I20260621 07:00:51.347774 16413 raft_consensus.cc:2229] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:00:51.348392 16413 raft_consensus.cc:2258] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:51.350739 16413 ts_tablet_manager.cc:1905] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:51.358150 16413 ts_tablet_manager.cc:1918] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:00:51.358409 16413 log.cc:1196] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/1d419e18fd6e4bcc8edca21a3a842598
I20260621 07:00:51.359202 16413 ts_tablet_manager.cc:1939] T 1d419e18fd6e4bcc8edca21a3a842598 P 4f51a9cc3fd142859784dbb4e794c396: Deleting consensus metadata
I20260621 07:00:51.361203 15869 catalog_manager.cc:4841] TS 4f51a9cc3fd142859784dbb4e794c396 (127.2.245.195:32907): tablet 1d419e18fd6e4bcc8edca21a3a842598 (table test-table [id=786c6b7fccca462c9df8e2d844f95203]) successfully deleted
I20260621 07:00:51.450078 16412 tablet_replica.cc:330] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: stopping tablet replica
I20260621 07:00:51.450495 16412 raft_consensus.cc:2229] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:00:51.450889 16412 raft_consensus.cc:2258] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:00:51.452795 16412 ts_tablet_manager.cc:1905] T 1d419e18fd6e4bcc8edca21a3a842598 P 61444532d00e479fa63cfc88f617a5ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:00:51.455120 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15921
W20260621 07:00:51.477649 15870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:39625 (error 108)
I20260621 07:00:51.478399 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16085
I20260621 07:00:51.480245 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.479228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38022)
0621 07:00:51.479867 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.479998 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":407,"threads_started":1}
W20260621 07:00:51.480540 15870 catalog_manager.cc:4584] TS 61444532d00e479fa63cfc88f617a5ca (127.2.245.193:39625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d419e18fd6e4bcc8edca21a3a842598: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
I20260621 07:00:51.500401 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.479327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.2.245.194:36751)
0621 07:00:51.479519 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.479525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.479844 (+ 319us) server_negotiation.cc:407] Connection header received
0621 07:00:51.479920 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.479925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.480010 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.480121 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:51.481844 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.482538 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.484453 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.484894 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.484986 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.493752 (+ 8766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:51.493769 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:51.493786 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:51.493823 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:51.497518 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.498215 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.498225 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.498229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.498394 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.498741 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.498747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.498751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.499242 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:51.499570 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.499747 (+ 177us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.500014 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260621 07:00:51.501173 16221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:51.534271 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.533748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38034)
0621 07:00:51.533984 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.534094 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260621 07:00:51.535480 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.535049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38036)
0621 07:00:51.535207 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.535309 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 07:00:51.592489 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16085
I20260621 07:00:51.596030 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.595510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38042)
0621 07:00:51.595737 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.595845 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:00:51.597280 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.596820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38050)
0621 07:00:51.597010 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.597105 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260621 07:00:51.609674 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16230
I20260621 07:00:51.625559 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.610579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.2.245.195:32907)
0621 07:00:51.610752 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:51.610756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:51.610773 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:00:51.610928 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:51.610931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:51.611058 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:51.611141 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:51.612797 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.613577 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.615609 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:51.616058 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:51.616152 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:51.619305 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:51.619316 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:51.619331 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:51.619369 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:51.622584 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.623507 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.623513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.623515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.623657 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:51.624060 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:51.624063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:51.624065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:51.624390 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:51.624708 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:51.625040 (+ 332us) server_negotiation.cc:299] Negotiation successful
0621 07:00:51.625188 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:00:51.626304 16355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:51.678048 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.677574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38060)
0621 07:00:51.677790 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.677891 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260621 07:00:51.679205 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.678748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38076)
0621 07:00:51.678999 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.679077 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260621 07:00:51.750212 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16230
I20260621 07:00:51.769554 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 15851
I20260621 07:00:51.770798 15914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:51.882056 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:51.881422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39625 (local address 127.0.0.1:38082)
0621 07:00:51.881588 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:51.881822 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 07:00:51.882252 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 15851
I20260621 07:00:51.903697 3031 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260621 07:00:52.238691 3031 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260621 07:00:52.238880 3031 mini_ranger.cc:66] Stopping Ranger...
I20260621 07:00:52.584147 3031 mini_ranger.cc:68] Stopped Ranger
2026-06-21 07:00:52.584 UTC [14903] LOG: received smart shutdown request
2026-06-21 07:00:52.587 UTC [14903] LOG: background worker "logical replication launcher" (PID 14912) exited with exit code 1
2026-06-21 07:00:52.587 UTC [14907] LOG: shutting down
2026-06-21 07:00:52.732 UTC [14903] LOG: database system is shut down
2026-06-21T07:00:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50609 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:00:52 dist-test-slave-9zdv krb5kdc[16429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:00:52 dist-test-slave-9zdv krb5kdc[16429](info): set up 2 sockets
Jun 21 07:00:52 dist-test-slave-9zdv krb5kdc[16429](info): commencing operation
krb5kdc: starting...
W20260621 07:00:53.951648 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:00:54 dist-test-slave-9zdv krb5kdc[16429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:00:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:00:54.180238 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43649
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:00:54.513001 16442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:54.513314 16442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:54.513403 16442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:54.520802 16442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:54.520908 16442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:54.520960 16442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:54.521019 16442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:54.521060 16442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:54.531059 16442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:54.532352 16442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:54.533638 16442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:54.545298 16447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:54.545357 16450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:54.545563 16448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:54.546411 16442 server_base.cc:1034] running on GCE node
I20260621 07:00:54.547323 16442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:54.548748 16442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:54.549952 16442 hybrid_clock.cc:648] HybridClock initialized: now 1782025254549873 us; error 61 us; skew 500 ppm
I20260621 07:00:54.550298 16442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:54 dist-test-slave-9zdv krb5kdc[16429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:54.553560 16442 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:00:54.555105 16442 webserver.cc:460] Webserver started at http://127.2.245.254:35513/ using document root <none> and password file <none>
I20260621 07:00:54.555863 16442 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:54.555998 16442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:54.556324 16442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:00:54.559875 16442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "ca98f35f2df9482f91e57a1227c79520"
format_stamp: "Formatted at 2026-06-21 07:00:54 on dist-test-slave-9zdv"
server_key: "39727c2ba28224c10688de61f16b7288"
server_key_iv: "55081066e6e2c7f55e544380d54f90e4"
server_key_version: "encryptionkey@0"
I20260621 07:00:54.560739 16442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "ca98f35f2df9482f91e57a1227c79520"
format_stamp: "Formatted at 2026-06-21 07:00:54 on dist-test-slave-9zdv"
server_key: "39727c2ba28224c10688de61f16b7288"
server_key_iv: "55081066e6e2c7f55e544380d54f90e4"
server_key_version: "encryptionkey@0"
I20260621 07:00:54.566012 16442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 07:00:54.569463 16457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:54.570883 16442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:00:54.571085 16442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "ca98f35f2df9482f91e57a1227c79520"
format_stamp: "Formatted at 2026-06-21 07:00:54 on dist-test-slave-9zdv"
server_key: "39727c2ba28224c10688de61f16b7288"
server_key_iv: "55081066e6e2c7f55e544380d54f90e4"
server_key_version: "encryptionkey@0"
I20260621 07:00:54.571280 16442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:54.600008 16442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:54.604219 16442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:54.604527 16442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:54.623392 16442 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43649
I20260621 07:00:54.623425 16508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43649 every 8 connection(s)
I20260621 07:00:54.624704 16442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:00:54.628480 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:54.632588 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16442
I20260621 07:00:54.632848 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:00:54.633657 3031 external_mini_cluster.cc:1428] Setting key 1358560188a80eeb2ca2f44bdb4158a2
I20260621 07:00:54.636276 16509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:00:54 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.253@KRBTEST.COM" created.
I20260621 07:00:54.659819 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.640259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.2.245.254:43649)
0621 07:00:54.642025 (+ 1766us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:54.642034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:54.642080 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:00:54.643650 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:54.643656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:54.643874 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:54.644079 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:54.645266 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:54.646467 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:54.647907 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:54.648448 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:54.648651 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:54.652261 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:54.652269 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:54.652281 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:54.652309 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:54.655142 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:54.655869 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:54.655881 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:54.655886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:54.656042 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:54.656441 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:54.656444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:54.656446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:54.656992 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:54.657188 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:54.657884 (+ 696us) server_negotiation.cc:299] Negotiation successful
0621 07:00:54.658287 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":1561,"spinlock_wait_cycles":66304,"thread_start_us":1353,"threads_started":1}
I20260621 07:00:54.659855 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.640257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43649 (local address 127.0.0.1:55792)
0621 07:00:54.641188 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.641234 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:54.643290 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:54.644092 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:54.644098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:54.644774 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:54.645084 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:54.645101 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:54.646623 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:54.646629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:54.647772 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:54.647783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:54.648684 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:54.648691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:54.648854 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:54.649528 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:54.649549 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:54.652011 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:54.655263 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:54.655272 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:54.655280 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:54.655691 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:54.656170 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:54.656174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:54.656176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:54.656338 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:54.657175 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:54.657184 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:54.657479 (+ 295us) client_negotiation.cc:769] Sending connection context
0621 07:00:54.657753 (+ 274us) client_negotiation.cc:240] Negotiation successful
0621 07:00:54.658383 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":180,"thread_start_us":442,"threads_started":1}
I20260621 07:00:54.663005 16509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:00:54.664222 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.663763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51174)
0621 07:00:54.663939 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.664068 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260621 07:00:54.664535 16461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
I20260621 07:00:54.666173 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.665724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51186)
0621 07:00:54.665904 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.665995 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260621 07:00:54.666368 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:46241: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
I20260621 07:00:54.684021 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:46241
--webserver_interface=127.2.245.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:00:54.715726 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } attempt: 1
I20260621 07:00:54.717015 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.716472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51194)
0621 07:00:54.716698 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.716847 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:00:54.718104 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.717676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51202)
0621 07:00:54.717871 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.717953 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260621 07:00:54.718321 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:46241: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
I20260621 07:00:54.786640 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } attempt: 2
I20260621 07:00:54.788038 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.787452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51214)
0621 07:00:54.787758 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.787869 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260621 07:00:54.789120 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.788724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51222)
0621 07:00:54.788891 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.788973 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260621 07:00:54.789326 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:46241: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
I20260621 07:00:54.880630 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } attempt: 3
I20260621 07:00:54.881929 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.881290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51230)
0621 07:00:54.881555 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.881726 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260621 07:00:54.883142 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:54.882603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51246)
0621 07:00:54.882800 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:54.882900 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260621 07:00:54.883355 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:46241: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
W20260621 07:00:55.002056 16516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:55.002379 16516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:55.002592 16516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:55.009990 16516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:55.010103 16516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:55.010216 16516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:55.010316 16516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:55.010386 16516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:55.020615 16516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:46241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.245.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:55.022002 16516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:55.023310 16516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:00:55.026631 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } attempt: 4
I20260621 07:00:55.027937 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.027300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51258)
0621 07:00:55.027664 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.027766 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260621 07:00:55.029130 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.028603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51268)
0621 07:00:55.028846 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.028974 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260621 07:00:55.029378 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:46241: Network error: Client connection negotiation failed: client connection to 127.2.245.253:46241: connect: Connection refused (error 111)
W20260621 07:00:55.035531 16521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:55.035636 16522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:55.035609 16524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:55.036316 16516 server_base.cc:1034] running on GCE node
I20260621 07:00:55.037051 16516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:55.038412 16516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:55.039638 16516 hybrid_clock.cc:648] HybridClock initialized: now 1782025255039608 us; error 52 us; skew 500 ppm
I20260621 07:00:55.039978 16516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:55.043032 16516 init.cc:377] Logged in from keytab as kudu/127.2.245.253@KRBTEST.COM (short username kudu)
I20260621 07:00:55.044500 16516 webserver.cc:460] Webserver started at http://127.2.245.253:38299/ using document root <none> and password file <none>
I20260621 07:00:55.045185 16516 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:55.045327 16516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:55.045629 16516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:00:55.048609 16516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data/instance:
uuid: "45fc7257db0c470287ab613d033ad25b"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "379004ca6e50dcebc7d7e45fa8040262"
server_key_iv: "2a11434a30f842ce83eb4484ccb6b7cd"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.049412 16516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal/instance:
uuid: "45fc7257db0c470287ab613d033ad25b"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "379004ca6e50dcebc7d7e45fa8040262"
server_key_iv: "2a11434a30f842ce83eb4484ccb6b7cd"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.054577 16516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 07:00:55.057950 16531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:55.059273 16516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:00:55.059449 16516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "45fc7257db0c470287ab613d033ad25b"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "379004ca6e50dcebc7d7e45fa8040262"
server_key_iv: "2a11434a30f842ce83eb4484ccb6b7cd"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.059672 16516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:55.092254 16516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:55.096275 16516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:55.096565 16516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:55.114777 16516 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.253:46241
I20260621 07:00:55.114800 16582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.253:46241 every 8 connection(s)
I20260621 07:00:55.116262 16516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
I20260621 07:00:55.119959 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:55.124724 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16516
I20260621 07:00:55.124908 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-1/wal/instance
I20260621 07:00:55.125336 3031 external_mini_cluster.cc:1428] Setting key 1dba2ee0447af6c1edfdce75822e2848
I20260621 07:00:55.126341 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.252@KRBTEST.COM; defaulting to no policy
I20260621 07:00:55.148612 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.129028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.2.245.254:43649)
0621 07:00:55.129303 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.129312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.130464 (+ 1152us) server_negotiation.cc:407] Connection header received
0621 07:00:55.132958 (+ 2494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.132966 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.133067 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.133191 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.134647 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.135803 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.137311 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.138095 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.138240 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.141638 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.141647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.141655 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.141701 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.145495 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.146247 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.146254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.146258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.146408 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.146813 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.146822 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.146827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.147269 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.147444 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.148131 (+ 687us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.148366 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260621 07:00:55.149642 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.129482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43649 (local address 127.0.0.1:55794)
0621 07:00:55.130263 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.130322 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.132688 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.133368 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.133375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.133909 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.134465 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.134481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.135953 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.135957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.137148 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.137160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.138319 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.138324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.138488 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.139241 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.139261 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.141411 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.145642 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.145648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.145684 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.146107 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.146531 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.146534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.146536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.146690 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.147393 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.147399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.147782 (+ 383us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.148019 (+ 237us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.148465 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":300,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.252@KRBTEST.COM" created.
I20260621 07:00:55.151625 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:00:55.166121 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.152480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51276)
0621 07:00:55.152641 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.152656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.152742 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.153821 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.153825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.154274 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.154399 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.154407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.155772 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.155777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.156609 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.156620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.157407 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.157412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.157680 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.158262 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.158279 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.160275 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.163623 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.163630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.163634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.164005 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.164378 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.164385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.164391 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.164545 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.165132 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.165137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.165272 (+ 135us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.165408 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.165693 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":43}
I20260621 07:00:55.166174 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.152777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.2.245.253:46241)
0621 07:00:55.153392 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.153397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.153421 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:00:55.153472 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.153478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.153668 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.153834 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.154558 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.155627 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.156747 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.157264 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.157399 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.160489 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.160500 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.160513 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.160547 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.163487 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.164144 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.164154 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.164157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.164271 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.164647 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.164655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.164658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.165003 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.165191 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.165548 (+ 357us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.165798 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":484,"thread_start_us":370,"threads_started":1}
I20260621 07:00:55.168503 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:00:55.169658 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.169180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40562)
0621 07:00:55.169371 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.169466 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260621 07:00:55.169891 16532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.171408 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.171054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40564)
0621 07:00:55.171223 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.171310 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260621 07:00:55.171641 16583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
I20260621 07:00:55.177568 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:39945
--webserver_interface=127.2.245.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:00:55.220932 16583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 1
I20260621 07:00:55.222311 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.221696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40574)
0621 07:00:55.221970 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.222101 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260621 07:00:55.223541 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.223055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40588)
0621 07:00:55.223260 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.223370 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260621 07:00:55.223805 16583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.292059 16583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 2
I20260621 07:00:55.293375 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.292775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40592)
0621 07:00:55.293013 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.293160 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260621 07:00:55.294739 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.294210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40606)
0621 07:00:55.294413 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.294519 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260621 07:00:55.295038 16583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.328667 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } attempt: 5
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:00:55.342295 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.329449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51282)
0621 07:00:55.329659 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.329683 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.329814 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.330163 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.330170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.330596 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.330772 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.330781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.331818 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.331824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.332716 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.332728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.333475 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.333481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.333715 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.334444 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.334461 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.336723 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.339929 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.339936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.339940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.340367 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.340815 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.340818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.340821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.340949 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.341605 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.341608 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.341739 (+ 131us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.341858 (+ 119us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.342061 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260621 07:00:55.342530 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.329601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.2.245.253:46241)
0621 07:00:55.329807 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.329813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.329836 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:00:55.329918 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.329923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.329996 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.330117 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.330961 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.331665 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.332881 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.333321 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.333457 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.336936 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.336941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.336944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.336976 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.339750 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.340522 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.340529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.340532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.340688 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.341059 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.341064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.341067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.341469 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.341622 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.341997 (+ 375us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.342236 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:00:55.343608 16509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:00:55.344763 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.344263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40618)
0621 07:00:55.344482 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.344588 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260621 07:00:55.345890 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.345430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40628)
0621 07:00:55.345633 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.345737 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260621 07:00:55.346171 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.386257 16583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 3
I20260621 07:00:55.387574 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.386943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40640)
0621 07:00:55.387264 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.387377 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260621 07:00:55.388732 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.388241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40650)
0621 07:00:55.388479 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.388572 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260621 07:00:55.388965 16583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.397433 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 1
I20260621 07:00:55.398607 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.398070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40666)
0621 07:00:55.398284 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.398402 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:00:55.399777 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.399319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40678)
0621 07:00:55.399522 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.399625 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260621 07:00:55.400005 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.468269 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 2
I20260621 07:00:55.469503 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.468931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40682)
0621 07:00:55.469191 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.469302 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:00:55.470623 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.470216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40696)
0621 07:00:55.470378 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.470472 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260621 07:00:55.470855 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
W20260621 07:00:55.509382 16590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:55.509684 16590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:55.509774 16590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:55.516978 16590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:55.517082 16590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:55.517128 16590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:55.517184 16590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:55.517227 16590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:55.527173 16590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43649,127.2.245.253:46241,127.2.245.252:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.245.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:55.528522 16590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:55.529773 16590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:00:55.532217 16583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 4
I20260621 07:00:55.533413 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.532860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40704)
0621 07:00:55.533104 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.533214 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260621 07:00:55.534488 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.534108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40716)
0621 07:00:55.534266 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.534354 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260621 07:00:55.534755 16583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
W20260621 07:00:55.542176 16595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:55.542565 16598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:55.542759 16596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:55.543743 16590 server_base.cc:1034] running on GCE node
I20260621 07:00:55.544518 16590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:55.545903 16590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:55.547125 16590 hybrid_clock.cc:648] HybridClock initialized: now 1782025255547086 us; error 34 us; skew 500 ppm
I20260621 07:00:55.547559 16590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:55.550694 16590 init.cc:377] Logged in from keytab as kudu/127.2.245.252@KRBTEST.COM (short username kudu)
I20260621 07:00:55.552273 16590 webserver.cc:460] Webserver started at http://127.2.245.252:45763/ using document root <none> and password file <none>
I20260621 07:00:55.552968 16590 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:55.553114 16590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:55.553437 16590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:00:55.556519 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data/instance:
uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "ec1b72de84bb26c1dcbb6af49fe7ac36"
server_key_iv: "1b212a76475fa246d689d1587b72eec7"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.557344 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal/instance:
uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "ec1b72de84bb26c1dcbb6af49fe7ac36"
server_key_iv: "1b212a76475fa246d689d1587b72eec7"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.562366 16590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 07:00:55.566185 16605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:55.567584 16590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 07:00:55.567806 16590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78"
format_stamp: "Formatted at 2026-06-21 07:00:55 on dist-test-slave-9zdv"
server_key: "ec1b72de84bb26c1dcbb6af49fe7ac36"
server_key_iv: "1b212a76475fa246d689d1587b72eec7"
server_key_version: "encryptionkey@0"
I20260621 07:00:55.568018 16590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:55.577122 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 3
I20260621 07:00:55.578225 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.577716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40720)
0621 07:00:55.577980 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.578074 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260621 07:00:55.579272 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.578862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40730)
0621 07:00:55.579108 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.579177 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260621 07:00:55.579489 16509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:39945: Network error: Client connection negotiation failed: client connection to 127.2.245.252:39945: connect: Connection refused (error 111)
I20260621 07:00:55.590817 16590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:55.595564 16590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:55.595855 16590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:55.614252 16590 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.252:39945
I20260621 07:00:55.614288 16656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.252:39945 every 8 connection(s)
I20260621 07:00:55.615626 16590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
I20260621 07:00:55.618786 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16590
I20260621 07:00:55.619016 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782025057166075-3031-0/minicluster-data/master-2/wal/instance
I20260621 07:00:55.619393 16657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:55.619552 3031 external_mini_cluster.cc:1428] Setting key c63158f4ae910cebf69140deb5cd861c
I20260621 07:00:55.629364 16657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:55.645435 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.621347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.2.245.252:39945)
0621 07:00:55.622124 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.622135 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.622178 (+ 43us) server_negotiation.cc:407] Connection header received
0621 07:00:55.624848 (+ 2670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.624877 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.625384 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.625796 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.627647 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.629418 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.631441 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.632474 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.632698 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.637378 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.637385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.637394 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.637427 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.640895 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.641725 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.641736 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.641745 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.641904 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.642258 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.642262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.642264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.642930 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.643273 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.643687 (+ 414us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.644106 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":367,"threads_started":1}
I20260621 07:00:55.647667 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.632219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.2.245.254:43649)
0621 07:00:55.632429 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.632435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.633191 (+ 756us) server_negotiation.cc:407] Connection header received
0621 07:00:55.633388 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.633394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.633478 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.633562 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.634649 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.635805 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.637314 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.637710 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.637797 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.641574 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.641579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.641582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.641615 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.644885 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.645587 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.645595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.645602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.645778 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.646166 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.646173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.646176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.646416 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.646613 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.647074 (+ 461us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.647286 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:00:55.647707 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.632299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43649 (local address 127.0.0.1:55808)
0621 07:00:55.633069 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.633109 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.633253 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.633662 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.633666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.634229 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.634477 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.634485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.635950 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.635956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.637148 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.637161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.637898 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.637906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.638043 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.638615 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.638644 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.641364 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.645013 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.645021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.645025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.645456 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.645909 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.645914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.645919 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.646047 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.646549 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.646556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.646742 (+ 186us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.646930 (+ 188us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.647272 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":394,"threads_started":1}
I20260621 07:00:55.649351 16657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:55.663264 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.649958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:46241 (local address 127.0.0.1:51290)
0621 07:00:55.650124 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.650140 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.650230 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.650672 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.650674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.651032 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.651164 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.651171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.652407 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.652412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.653763 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.653774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.654709 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.654718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.654903 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.655452 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.655476 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.657972 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.660928 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.660933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.660936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.661433 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.661774 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.661779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.661782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.661942 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.662377 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.662382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.662528 (+ 146us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.662685 (+ 157us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.662893 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260621 07:00:55.663497 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.650159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.2.245.253:46241)
0621 07:00:55.650370 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.650378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.650399 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:00:55.650459 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.650464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.650536 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.650652 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.651306 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.652277 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.653890 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.654549 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.654646 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.658191 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.658197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.658200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.658231 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.660809 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.661545 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.661548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.661550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.661666 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.662044 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.662046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.662048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.662266 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.662403 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.662778 (+ 375us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.663033 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260621 07:00:55.664613 16657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:00:55.665355 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.648941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.2.245.254:43649)
0621 07:00:55.649106 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.649123 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.649143 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:00:55.649190 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.649195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.649257 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.649350 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.650854 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.651946 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.653663 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.654216 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.654336 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.659171 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.659179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.659187 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.659215 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.662277 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.663090 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.663096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.663100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.663238 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.663846 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.663851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.663856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.664196 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.664470 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.664797 (+ 327us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.665000 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:00:55.678637 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.665315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40750)
0621 07:00:55.665456 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.665482 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.665586 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.665899 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.665903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.666255 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.666370 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.666378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.667515 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.667519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.668905 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.668919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.669826 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.669831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.669964 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.670466 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.670487 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.673084 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.676146 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.676149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.676152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.676539 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.676925 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.676928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.676930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.677047 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.677632 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.677636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.677771 (+ 135us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.677907 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.678240 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260621 07:00:55.678716 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.665503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.2.245.252:39945)
0621 07:00:55.665656 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.665664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.665680 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:00:55.665741 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.665744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.665789 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.665884 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.666528 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.667355 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.669072 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.669680 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.669794 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.673234 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.673242 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.673251 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.673275 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.676011 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.676655 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.676661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.676664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.676803 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.677152 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.677158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.677161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.677515 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.677651 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.678024 (+ 373us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.678398 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":51}
I20260621 07:00:55.682821 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.667143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.2.245.253:46241)
0621 07:00:55.667283 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.667288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.667304 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:00:55.667445 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.667452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.667521 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.667602 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.669109 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.670074 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.671911 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.672604 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.672805 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.677452 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.677456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.677459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.677486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.680462 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.681225 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.681228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.681230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.681343 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.681696 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.681699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.681700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.681933 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.682110 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.682419 (+ 309us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.682588 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:00:55.684732 16657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: Bootstrap starting.
I20260621 07:00:55.688660 16657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:55.690042 16657 log.cc:824] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:55.691080 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:00:55.693372 16657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: No bootstrap required, opened a new log
I20260621 07:00:55.706070 16657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.706602 16657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:55.706709 16657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a3f356bf0b848c1aa3250a0b8fcfa78, State: Initialized, Role: FOLLOWER
I20260621 07:00:55.708395 16657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.715389 16670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.715811 16670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:55.717705 16657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:55.718766 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.696661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.2.245.253:46241)
0621 07:00:55.696836 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.696842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.697810 (+ 968us) server_negotiation.cc:407] Connection header received
0621 07:00:55.698355 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.698365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.698463 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.698561 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.701738 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.703177 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.705678 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.706392 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.706514 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.711083 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.711093 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.711101 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.711159 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.715232 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.716308 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.716314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.716317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.716464 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.716957 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.716962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.716965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.717326 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.717534 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.718174 (+ 640us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.718416 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:00:55.722100 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.696643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.2.245.254:43649)
0621 07:00:55.696874 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.696882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.703577 (+ 6695us) server_negotiation.cc:407] Connection header received
0621 07:00:55.703738 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.703744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.703826 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.703923 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.706414 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.707528 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.709897 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.710535 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.710625 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.714459 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.714467 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.714472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.714509 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.718027 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.718871 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.718877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.718883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.719108 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.719658 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.719663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.719667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.720031 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.720254 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.721265 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.721580 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:00:55.724691 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.697789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.2.245.252:39945)
0621 07:00:55.698023 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.698032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.707541 (+ 9509us) server_negotiation.cc:407] Connection header received
0621 07:00:55.707745 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.707753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.707830 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.707928 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.709948 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.710994 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.713256 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.714013 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.714113 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.718230 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.718242 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.718247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.718290 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.721914 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.722646 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.722652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.722655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.722796 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.723232 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.723238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.723242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.723592 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.723840 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.724141 (+ 301us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.724346 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260621 07:00:55.734862 16681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:00:55.735023 16681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:00:55.730733 16590 master_runner.cc:186] Error getting master registration for 127.2.245.254:43649: OK, instance_id { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" instance_seqno: 1782025254617983 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:00:55.740535 16590 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:00:55.756695 16509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 4
I20260621 07:00:55.763408 16670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:00:55.763737 16670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025254, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:00:55.765323 16670 leader_election.cc:288] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca98f35f2df9482f91e57a1227c79520 (127.2.245.254:43649), 45fc7257db0c470287ab613d033ad25b (127.2.245.253:46241)
I20260621 07:00:55.773643 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.757275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40774)
0621 07:00:55.757499 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.757517 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.757634 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.758171 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.758176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.758723 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.758935 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.758971 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.760252 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.760258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.761766 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.761781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.762504 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.762511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.762774 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.763502 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.763526 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.766354 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.770683 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.770689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.770693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.771232 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.771663 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.771669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.771673 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.771900 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.772516 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.772523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.772719 (+ 196us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.772926 (+ 207us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.773217 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260621 07:00:55.773662 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.757344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40774 (local address 127.2.245.252:39945)
0621 07:00:55.757485 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.757489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.757596 (+ 107us) server_negotiation.cc:407] Connection header received
0621 07:00:55.757796 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.757802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.757900 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.757981 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.759124 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.760112 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.761909 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.762360 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.762453 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.766560 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.766569 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.766576 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.766636 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.770553 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.771346 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.771351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.771355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.771535 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.772011 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.772016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.772019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.772363 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.772505 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.773047 (+ 542us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.773281 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260621 07:00:55.779790 16509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: Bootstrap starting.
I20260621 07:00:55.783931 16509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:55.785270 16509 log.cc:824] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:55.788928 16509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: No bootstrap required, opened a new log
I20260621 07:00:55.800102 16509 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.800496 16509 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:55.800618 16509 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca98f35f2df9482f91e57a1227c79520, State: Initialized, Role: FOLLOWER
I20260621 07:00:55.801283 16509 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.802260 16684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.802659 16684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:55.803563 16509 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:55.803565 16484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca98f35f2df9482f91e57a1227c79520" is_pre_election: true
I20260621 07:00:55.804152 16484 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a3f356bf0b848c1aa3250a0b8fcfa78 in term 0.
I20260621 07:00:55.804775 16606 leader_election.cc:302] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a3f356bf0b848c1aa3250a0b8fcfa78, ca98f35f2df9482f91e57a1227c79520; no voters:
I20260621 07:00:55.805302 16670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:00:55.805457 16670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:00:55.805617 16670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:55.808861 16670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.810606 16484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca98f35f2df9482f91e57a1227c79520"
I20260621 07:00:55.810815 16484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:55.811056 16670 leader_election.cc:288] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [CANDIDATE]: Term 1 election: Requested vote from peers ca98f35f2df9482f91e57a1227c79520 (127.2.245.254:43649), 45fc7257db0c470287ab613d033ad25b (127.2.245.253:46241)
I20260621 07:00:55.814292 16484 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a3f356bf0b848c1aa3250a0b8fcfa78 in term 1.
I20260621 07:00:55.814786 16606 leader_election.cc:302] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a3f356bf0b848c1aa3250a0b8fcfa78, ca98f35f2df9482f91e57a1227c79520; no voters:
I20260621 07:00:55.815151 16670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:00:55.815623 16670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [term 1 LEADER]: Becoming Leader. State: Replica: 8a3f356bf0b848c1aa3250a0b8fcfa78, State: Running, Role: LEADER
I20260621 07:00:55.816192 16670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.819500 16689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a3f356bf0b848c1aa3250a0b8fcfa78. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.819953 16689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:55.823705 16691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:00:55.825886 16691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:00:55.833971 16583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } attempt: 5
I20260621 07:00:55.844514 16484 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [term 1 FOLLOWER]: Refusing update from remote peer 8a3f356bf0b848c1aa3250a0b8fcfa78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:55.845474 16689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 07:00:55.847803 16611 tablet.cc:2369] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 21 07:00:55 dist-test-slave-9zdv krb5kdc[16429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025255, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:00:55.865547 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.834687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:39945 (local address 127.0.0.1:40790)
0621 07:00:55.835156 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0621 07:00:55.835179 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 07:00:55.835311 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:00:55.836607 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:00:55.836611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:00:55.837151 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:00:55.837354 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.837366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.838879 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.838886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.840260 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:00:55.840275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.841468 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.841474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:00:55.841723 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.842422 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:00:55.842469 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:00:55.851799 (+ 9330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:00:55.856606 (+ 4807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.856612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.856615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.857240 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.857853 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:00:55.857861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:00:55.857867 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:00:55.858407 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:00:55.860292 (+ 1885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:00:55.860300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:00:55.860714 (+ 414us) client_negotiation.cc:769] Sending connection context
0621 07:00:55.861222 (+ 508us) client_negotiation.cc:240] Negotiation successful
0621 07:00:55.863134 (+ 1912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292}
I20260621 07:00:55.866381 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:55.835804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.2.245.252:39945)
0621 07:00:55.836017 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:55.836035 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:55.836059 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:00:55.836136 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:55.836145 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:55.836240 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:55.836356 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:55.837576 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.838624 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.840469 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:55.841129 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:55.841251 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:55.852188 (+ 10937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:55.852198 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:55.852205 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:55.852276 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:55.856411 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.857458 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.857467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.857473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.857653 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:55.858627 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:55.858636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:55.858641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:55.859983 (+ 1342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:55.861611 (+ 1628us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:55.862381 (+ 770us) server_negotiation.cc:299] Negotiation successful
0621 07:00:55.863941 (+ 1560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260621 07:00:55.875276 16583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: Bootstrap starting.
W20260621 07:00:55.879462 16463 tablet.cc:2369] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:00:55.890797 16684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a3f356bf0b848c1aa3250a0b8fcfa78. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.891178 16684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:55.891440 16695 mvcc.cc:205] Tried to move back new op lower bound from 7299175447929155584 to 7299175447827312640. Current Snapshot: MvccSnapshot[applied={T|T < 7299175447929155584}]
I20260621 07:00:55.892897 16670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.893220 16670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:55.893810 16583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:55.896008 16583 log.cc:824] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:55.896838 16684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.897094 16684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:55.899447 16691 catalog_manager.cc:1347] Generated new cluster ID: af1de1b121f44e1c961c35566e96e4d9
I20260621 07:00:55.899627 16691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260621 07:00:55.928188 16702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:00:55.928328 16702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:00:55.930305 16583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: No bootstrap required, opened a new log
I20260621 07:00:55.911480 16442 master_runner.cc:186] Error getting master registration for 127.2.245.253:46241: OK, instance_id { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" instance_seqno: 1782025255109512 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:00:55.934831 16442 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:00:55.947410 16583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.948268 16583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:55.948549 16583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fc7257db0c470287ab613d033ad25b, State: Initialized, Role: FOLLOWER
I20260621 07:00:55.950183 16583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.956744 16691 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:00:55.958647 16691 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:00:55.972081 16706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:55.972612 16706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:55.975883 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fc7257db0c470287ab613d033ad25b" is_pre_election: true
I20260621 07:00:55.976652 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fc7257db0c470287ab613d033ad25b"
I20260621 07:00:55.977546 16556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:55.980829 16691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78: Generated new TSK 0
I20260621 07:00:55.981870 16691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:00:55.985050 16556 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 1 FOLLOWER]: Refusing update from remote peer 8a3f356bf0b848c1aa3250a0b8fcfa78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:00:55.985653 16706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:00:55.985934 16706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } }
I20260621 07:00:55.986186 16689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:00:55.987725 16706 leader_election.cc:288] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca98f35f2df9482f91e57a1227c79520 (127.2.245.254:43649), 8a3f356bf0b848c1aa3250a0b8fcfa78 (127.2.245.252:39945)
I20260621 07:00:55.990907 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45fc7257db0c470287ab613d033ad25b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" is_pre_election: true
I20260621 07:00:55.992440 16484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45fc7257db0c470287ab613d033ad25b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca98f35f2df9482f91e57a1227c79520" is_pre_election: true
I20260621 07:00:55.993085 16532 leader_election.cc:302] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45fc7257db0c470287ab613d033ad25b; no voters: 8a3f356bf0b848c1aa3250a0b8fcfa78, ca98f35f2df9482f91e57a1227c79520
I20260621 07:00:55.994705 16583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:56.034970 16706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a3f356bf0b848c1aa3250a0b8fcfa78. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:56.035310 16706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:56.035846 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:56.036325 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:00:56.037039 16708 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:00:56.084728 16726 catalog_manager.cc:1259] Loaded cluster ID: af1de1b121f44e1c961c35566e96e4d9
I20260621 07:00:56.084944 16726 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: loading cluster ID for follower catalog manager: success
I20260621 07:00:56.093057 16726 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: acquiring CA information for follower catalog manager: success
I20260621 07:00:56.095842 16726 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45fc7257db0c470287ab613d033ad25b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:00:56.385527 16689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98f35f2df9482f91e57a1227c79520" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43649 } } peers { permanent_uuid: "45fc7257db0c470287ab613d033ad25b" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 46241 } } peers { permanent_uuid: "8a3f356bf0b848c1aa3250a0b8fcfa78" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 39945 } } }
I20260621 07:00:56.385890 16689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a3f356bf0b848c1aa3250a0b8fcfa78 [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:56.930222 16702 catalog_manager.cc:1259] Loaded cluster ID: af1de1b121f44e1c961c35566e96e4d9
I20260621 07:00:56.930320 16702 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: loading cluster ID for follower catalog manager: success
I20260621 07:00:56.932905 16702 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: acquiring CA information for follower catalog manager: success
I20260621 07:00:56.934689 16702 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca98f35f2df9482f91e57a1227c79520: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:00:57.123723 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16442
I20260621 07:00:57.125007 16504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:57.236338 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16442
I20260621 07:00:57.256637 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16516
I20260621 07:00:57.257769 16578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:57.367604 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16516
I20260621 07:00:57.387408 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16590
I20260621 07:00:57.388480 16652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:57.497318 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16590
2026-06-21T07:00:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4639 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:00:57 dist-test-slave-9zdv krb5kdc[16736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:00:57 dist-test-slave-9zdv krb5kdc[16736](info): set up 2 sockets
Jun 21 07:00:57 dist-test-slave-9zdv krb5kdc[16736](info): commencing operation
krb5kdc: starting...
W20260621 07:00:58.587610 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:00:58 dist-test-slave-9zdv krb5kdc[16736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:00:58.814193 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:41581
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:41581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:00:59.138666 16749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:00:59.139036 16749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:00:59.139195 16749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:00:59.146262 16749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:00:59.146412 16749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:00:59.146529 16749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:00:59.146636 16749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:00:59.146723 16749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:00:59.156667 16749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:41581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:41581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:00:59.158041 16749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:00:59.159337 16749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:00:59.170971 16757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:59.170879 16755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:00:59.171015 16754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:00:59.171954 16749 server_base.cc:1034] running on GCE node
I20260621 07:00:59.172726 16749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:00:59.174207 16749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:00:59.175447 16749 hybrid_clock.cc:648] HybridClock initialized: now 1782025259175372 us; error 70 us; skew 500 ppm
I20260621 07:00:59.175794 16749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:00:59 dist-test-slave-9zdv krb5kdc[16736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025259, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:00:59.178889 16749 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:00:59.180383 16749 webserver.cc:460] Webserver started at http://127.2.245.254:40291/ using document root <none> and password file <none>
I20260621 07:00:59.181083 16749 fs_manager.cc:359] Metadata directory not provided
I20260621 07:00:59.181232 16749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:00:59.181553 16749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:00:59.184613 16749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27"
format_stamp: "Formatted at 2026-06-21 07:00:59 on dist-test-slave-9zdv"
server_key: "30f7af549580e83e5f9a332e650a69ed"
server_key_iv: "42b2f00a7d6729b3988e3a135da199bd"
server_key_version: "encryptionkey@0"
I20260621 07:00:59.185367 16749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27"
format_stamp: "Formatted at 2026-06-21 07:00:59 on dist-test-slave-9zdv"
server_key: "30f7af549580e83e5f9a332e650a69ed"
server_key_iv: "42b2f00a7d6729b3988e3a135da199bd"
server_key_version: "encryptionkey@0"
I20260621 07:00:59.190527 16749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:00:59.194007 16764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:00:59.195515 16749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:00:59.195677 16749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27"
format_stamp: "Formatted at 2026-06-21 07:00:59 on dist-test-slave-9zdv"
server_key: "30f7af549580e83e5f9a332e650a69ed"
server_key_iv: "42b2f00a7d6729b3988e3a135da199bd"
server_key_version: "encryptionkey@0"
I20260621 07:00:59.195843 16749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:00:59.211356 16749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:00:59.215691 16749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:00:59.215914 16749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:00:59.233973 16749 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:41581
I20260621 07:00:59.234006 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:41581 every 8 connection(s)
I20260621 07:00:59.235493 16749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:00:59.239261 16816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:00:59.245848 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16749
I20260621 07:00:59.246060 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:00:59.246722 3031 external_mini_cluster.cc:1428] Setting key 1add857ebfaac21475b019044f2043c7
I20260621 07:00:59.250042 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27: Bootstrap starting.
I20260621 07:00:59.253983 16816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27: Neither blocks nor log segments found. Creating new log.
I20260621 07:00:59.255239 16816 log.cc:824] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27: Log is configured to *not* fsync() on all Append() calls
I20260621 07:00:59.258540 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27: No bootstrap required, opened a new log
Jun 21 07:00:59 dist-test-slave-9zdv krb5kdc[16736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:00:59.267416 16816 raft_consensus.cc:348] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } }
I20260621 07:00:59.267792 16816 raft_consensus.cc:374] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:00:59.267895 16816 raft_consensus.cc:729] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aacd5089a4ec4bfe9e44daaf93fb8f27, State: Initialized, Role: FOLLOWER
I20260621 07:00:59.268603 16816 consensus_queue.cc:260] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } }
I20260621 07:00:59.268900 16816 raft_consensus.cc:388] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:00:59.269062 16816 raft_consensus.cc:482] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:00:59.269253 16816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:00:59.272116 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:00:59.248742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.2.245.254:41581)
0621 07:00:59.249544 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0621 07:00:59.249560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:00:59.249613 (+ 53us) server_negotiation.cc:407] Connection header received
0621 07:00:59.252068 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:00:59.252113 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:00:59.252626 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:00:59.253122 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:00:59.254528 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:59.256376 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:59.258296 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:00:59.259117 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:00:59.259264 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:00:59.264149 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:00:59.264157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:00:59.264171 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:00:59.264212 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:00:59.267397 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:59.268361 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:59.268368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:59.268376 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:59.268586 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:00:59.268999 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:00:59.269003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:00:59.269006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:00:59.269733 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:00:59.270036 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0621 07:00:59.270366 (+ 330us) server_negotiation.cc:299] Negotiation successful
0621 07:00:59.270803 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":343,"threads_started":1}
I20260621 07:00:59.272409 16816 raft_consensus.cc:504] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } }
I20260621 07:00:59.272969 16816 leader_election.cc:302] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aacd5089a4ec4bfe9e44daaf93fb8f27; no voters:
I20260621 07:00:59.273676 16816 leader_election.cc:288] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:00:59.273815 16821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:00:59.275065 16821 raft_consensus.cc:686] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [term 1 LEADER]: Becoming Leader. State: Replica: aacd5089a4ec4bfe9e44daaf93fb8f27, State: Running, Role: LEADER
I20260621 07:00:59.275681 16821 consensus_queue.cc:237] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } }
I20260621 07:00:59.275883 16816 sys_catalog.cc:564] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:00:59.276700 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:00:59.278637 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } } }
I20260621 07:00:59.279031 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:59.279579 16823 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aacd5089a4ec4bfe9e44daaf93fb8f27. Latest consensus state: current_term: 1 leader_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd5089a4ec4bfe9e44daaf93fb8f27" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 41581 } } }
I20260621 07:00:59.279889 16823 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27 [sys.catalog]: This master's current role is: LEADER
I20260621 07:00:59.282398 16827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:00:59.286763 16827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:00:59.295017 16827 catalog_manager.cc:1347] Generated new cluster ID: ed1ec6e326204179bbf007000cc6a751
I20260621 07:00:59.295130 16827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:00:59.307469 16827 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:00:59.308710 16827 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:00:59.317806 16827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aacd5089a4ec4bfe9e44daaf93fb8f27: Generated new TSK 0
I20260621 07:00:59.318584 16827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:00:59.338960 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16749
I20260621 07:00:59.340257 16811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:00:59.432548 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16749
2026-06-21T07:00:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1931 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:00:59 dist-test-slave-9zdv krb5kdc[16845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:00:59 dist-test-slave-9zdv krb5kdc[16845](info): set up 2 sockets
Jun 21 07:00:59 dist-test-slave-9zdv krb5kdc[16845](info): commencing operation
krb5kdc: starting...
W20260621 07:01:00.531667 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:01:00 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:01:00.761734 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:01.086604 16858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:01.086959 16858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:01.087148 16858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:01.094556 16858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:01:01.094710 16858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:01.094810 16858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:01.094911 16858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:01:01.095011 16858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:01:01.105283 16858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:01.106650 16858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:01.107978 16858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:01.119868 16863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:01.119935 16864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:01.119812 16866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:01.120566 16858 server_base.cc:1034] running on GCE node
I20260621 07:01:01.121322 16858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:01.122641 16858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:01.123864 16858 hybrid_clock.cc:648] HybridClock initialized: now 1782025261123815 us; error 41 us; skew 500 ppm
I20260621 07:01:01.124166 16858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:01 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025261, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:01.131333 16858 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:01:01.132929 16858 webserver.cc:460] Webserver started at http://127.2.245.254:41669/ using document root <none> and password file <none>
I20260621 07:01:01.133592 16858 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:01.133701 16858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:01.133948 16858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:01.137007 16858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "631dc01aaf9d4e9b819863428136d264"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "5c1113167e9dc96c924e9825e0e69821"
server_key_iv: "71eec2470d8c64c691df7f5c87057e90"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.137718 16858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "631dc01aaf9d4e9b819863428136d264"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "5c1113167e9dc96c924e9825e0e69821"
server_key_iv: "71eec2470d8c64c691df7f5c87057e90"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.142930 16858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 07:01:01.146359 16873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:01.147929 16858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:01:01.148067 16858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "631dc01aaf9d4e9b819863428136d264"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "5c1113167e9dc96c924e9825e0e69821"
server_key_iv: "71eec2470d8c64c691df7f5c87057e90"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.148228 16858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:01.194586 16858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:01.199209 16858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:01.199462 16858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:01.218012 16858 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43509
I20260621 07:01:01.218044 16924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43509 every 8 connection(s)
I20260621 07:01:01.219509 16858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:01:01.223353 16925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:01.223374 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16858
I20260621 07:01:01.223595 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:01:01.224241 3031 external_mini_cluster.cc:1428] Setting key 763b393c54b7e346b864b20fcaccb20b
I20260621 07:01:01.237761 16925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap starting.
Jun 21 07:01:01 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:01.241093 16925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:01.242209 16925 log.cc:824] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:01.245003 16925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: No bootstrap required, opened a new log
I20260621 07:01:01.248601 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:01.226365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.2.245.254:43509)
0621 07:01:01.227202 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:01.227218 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:01.227272 (+ 54us) server_negotiation.cc:407] Connection header received
0621 07:01:01.230011 (+ 2739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:01.230042 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:01.230577 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:01.231099 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:01.232398 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.234384 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.235857 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.236924 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.237113 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:01.241416 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:01.241424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:01.241442 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:01.241488 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:01.244552 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:01.245262 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:01.245272 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:01.245283 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:01.245454 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:01.245846 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:01.245851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:01.245854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:01.246380 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:01.246776 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:01.247177 (+ 401us) server_negotiation.cc:299] Negotiation successful
0621 07:01:01.247613 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":338,"threads_started":1}
I20260621 07:01:01.253654 16925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:01.254043 16925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:01.254168 16925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Initialized, Role: FOLLOWER
I20260621 07:01:01.254691 16925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:01.254923 16925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:01:01.255117 16925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:01:01.255290 16925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:01.257674 16925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:01.258185 16925 leader_election.cc:302] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631dc01aaf9d4e9b819863428136d264; no voters:
I20260621 07:01:01.258798 16925 leader_election.cc:288] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:01:01.258961 16930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:01.259747 16930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 LEADER]: Becoming Leader. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Running, Role: LEADER
I20260621 07:01:01.260314 16930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:01.261066 16925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:01:01.263183 16932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631dc01aaf9d4e9b819863428136d264. Latest consensus state: current_term: 1 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:01.263548 16932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:01.263998 16931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:01.264364 16931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:01.264684 16937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:01:01.271463 16937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:01:01.281352 16937 catalog_manager.cc:1347] Generated new cluster ID: 620def677f10468e917425330d473a14
I20260621 07:01:01.281486 16937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:01:01.313566 16937 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:01:01.315577 16937 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:01:01.328567 16937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Generated new TSK 0
I20260621 07:01:01.329672 16937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:01:01.400385 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:01.739856 16951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:01.740170 16951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:01.740331 16951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:01.747658 16951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:01.747819 16951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:01.747978 16951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:01:01.757752 16951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:01.759058 16951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:01.760409 16951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:01.773495 16956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:01.773473 16959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:01.773445 16957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:01.774243 16951 server_base.cc:1034] running on GCE node
I20260621 07:01:01.774777 16951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:01.775677 16951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:01.776867 16951 hybrid_clock.cc:648] HybridClock initialized: now 1782025261776815 us; error 52 us; skew 500 ppm
I20260621 07:01:01.777164 16951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:01 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025261, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:01.780247 16951 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:01:01.781647 16951 webserver.cc:460] Webserver started at http://127.2.245.193:39059/ using document root <none> and password file <none>
I20260621 07:01:01.782325 16951 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:01.782464 16951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:01.782768 16951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:01.785676 16951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "9dca01673dd140289249555cd29d9f22"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "2c3087147a9239f6cb84b4859b3802cc"
server_key_iv: "bc73fc54a460d437436e56f87d6af35d"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.786449 16951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "9dca01673dd140289249555cd29d9f22"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "2c3087147a9239f6cb84b4859b3802cc"
server_key_iv: "bc73fc54a460d437436e56f87d6af35d"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.792116 16951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260621 07:01:01.795475 16966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:01.796696 16951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:01:01.796880 16951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "9dca01673dd140289249555cd29d9f22"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "2c3087147a9239f6cb84b4859b3802cc"
server_key_iv: "bc73fc54a460d437436e56f87d6af35d"
server_key_version: "encryptionkey@0"
I20260621 07:01:01.797083 16951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:01.811215 16951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:01.814827 16951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:01.815143 16951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:01.816028 16951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:01.817291 16951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:01.817394 16951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:01.817561 16951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:01.817636 16951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:01.850760 16951 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36487
I20260621 07:01:01.850795 17078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36487 every 8 connection(s)
I20260621 07:01:01.852146 16951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:01:01.861304 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 16951
I20260621 07:01:01.861512 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:01:01.862341 3031 external_mini_cluster.cc:1428] Setting key 061aad3e50b813dce1ae9eafb11228e6
Jun 21 07:01:01 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025261, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:01.872959 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:01.854547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:35611 (local address 127.2.245.254:43509)
0621 07:01:01.855054 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:01.855058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:01.855783 (+ 725us) server_negotiation.cc:407] Connection header received
0621 07:01:01.857592 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:01.857598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:01.857689 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:01.857809 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:01.859298 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.860041 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.861692 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.862434 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.862584 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:01.865936 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:01.865944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:01.865953 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:01.865996 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:01.869567 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:01.870504 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:01.870511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:01.870517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:01.870650 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:01.871198 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:01.871202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:01.871204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:01.871468 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:01.871574 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:01.872453 (+ 879us) server_negotiation.cc:299] Negotiation successful
0621 07:01:01.872713 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":267,"threads_started":1}
I20260621 07:01:01.874323 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:01.854919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:35611)
0621 07:01:01.855600 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:01.855634 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:01.857302 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:01.858041 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:01.858049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:01.858519 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:01.859135 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:01.859154 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.860196 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.860204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:01.861517 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:01.861564 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:01.862647 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:01.862652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:01.862860 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:01.863594 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:01.863616 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:01.865723 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:01.869730 (+ 4007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:01.869741 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:01.869759 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:01.870361 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:01.870785 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:01.870793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:01.870799 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:01.871072 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:01.871568 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:01.871578 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:01.872050 (+ 472us) client_negotiation.cc:769] Sending connection context
0621 07:01:01.872341 (+ 291us) client_negotiation.cc:240] Negotiation successful
0621 07:01:01.872997 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":245,"threads_started":1}
I20260621 07:01:01.875942 17079 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
I20260621 07:01:01.876238 17079 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:01.876951 17079 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
I20260621 07:01:01.879217 16890 ts_manager.cc:194] Registered new tserver with Master: 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:01.881351 16890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:35611
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 07:01:01.914351 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:02.238282 17085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:02.238600 17085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:02.238782 17085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:02.246271 17085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:02.246434 17085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:02.246629 17085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:01:02.256457 17085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:02.257740 17085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:02.259116 17085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:02.271827 17091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:02.271819 17090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:02.272090 17093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:02.272886 17085 server_base.cc:1034] running on GCE node
I20260621 07:01:02.273483 17085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:02.274466 17085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:02.275707 17085 hybrid_clock.cc:648] HybridClock initialized: now 1782025262275639 us; error 58 us; skew 500 ppm
I20260621 07:01:02.276006 17085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:02 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025262, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:02.279218 17085 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:01:02.280582 17085 webserver.cc:460] Webserver started at http://127.2.245.194:33477/ using document root <none> and password file <none>
I20260621 07:01:02.281312 17085 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:02.281471 17085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:02.281797 17085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:02.284642 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "97361611225943309ca2ae08922d3f81"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "511269161b76b6416371522bccb0ae8d"
server_key_iv: "6aeff80d1dc5c22b0bb25a0c5b25d543"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.285401 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "97361611225943309ca2ae08922d3f81"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "511269161b76b6416371522bccb0ae8d"
server_key_iv: "6aeff80d1dc5c22b0bb25a0c5b25d543"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.292788 17085 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260621 07:01:02.297050 17100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.298434 17085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:01:02.298559 17085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "97361611225943309ca2ae08922d3f81"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "511269161b76b6416371522bccb0ae8d"
server_key_iv: "6aeff80d1dc5c22b0bb25a0c5b25d543"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.298728 17085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:02.343106 17085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:02.347163 17085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:02.347409 17085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:02.348294 17085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:02.349547 17085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:02.349627 17085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.349712 17085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:02.349756 17085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.383474 17085 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38153
I20260621 07:01:02.383515 17212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38153 every 8 connection(s)
I20260621 07:01:02.384769 17085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:01:02.386130 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17085
I20260621 07:01:02.386307 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:01:02.386893 3031 external_mini_cluster.cc:1428] Setting key 7b38433c315c9c6b495b7801e69a84a7
Jun 21 07:01:02 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025262, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:02.410494 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.387588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:54351 (local address 127.2.245.254:43509)
0621 07:01:02.388069 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.388073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.389068 (+ 995us) server_negotiation.cc:407] Connection header received
0621 07:01:02.393863 (+ 4795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.393867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.393936 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.394031 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:02.396145 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.396911 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.398467 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.399249 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.399387 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.402984 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:02.402994 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:02.403002 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:02.403047 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:02.406856 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.407683 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.407690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.407696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.407859 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.408290 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.408297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.408302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.408702 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:02.408882 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.409665 (+ 783us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.409942 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":233,"threads_started":1}
I20260621 07:01:02.411166 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.388143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:54351)
0621 07:01:02.388939 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:02.388977 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:02.393530 (+ 4553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:02.394323 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:02.394337 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:02.395172 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:02.395965 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.395992 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.397072 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.397085 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.398309 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.398321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.399451 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.399456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.399707 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.400426 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:02.400460 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:02.402693 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:02.407060 (+ 4367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:02.407067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:02.407080 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:02.407543 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:02.407972 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:02.407975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:02.407977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:02.408162 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:02.408880 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:02.408889 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:02.409259 (+ 370us) client_negotiation.cc:769] Sending connection context
0621 07:01:02.409508 (+ 249us) client_negotiation.cc:240] Negotiation successful
0621 07:01:02.409929 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":297,"threads_started":1}
I20260621 07:01:02.412912 17213 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 07:01:02.413278 17213 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:02.414100 17213 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
I20260621 07:01:02.415779 16890 ts_manager.cc:194] Registered new tserver with Master: 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153)
I20260621 07:01:02.416586 16890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:54351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:01:02.441974 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:02.758960 17219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:02.759305 17219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:02.759476 17219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:02.766638 17219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:02.766788 17219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:02.766978 17219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:01:02.776217 17219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:02.777462 17219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:02.778743 17219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:02.791759 17224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:02.791718 17227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:02.791787 17225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:02.792549 17219 server_base.cc:1034] running on GCE node
I20260621 07:01:02.793135 17219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:02.794039 17219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:02.795275 17219 hybrid_clock.cc:648] HybridClock initialized: now 1782025262795220 us; error 59 us; skew 500 ppm
I20260621 07:01:02.795571 17219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:02 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025262, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:02.798705 17219 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:01:02.800192 17219 webserver.cc:460] Webserver started at http://127.2.245.195:34663/ using document root <none> and password file <none>
I20260621 07:01:02.800956 17219 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:02.801056 17219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:02.801365 17219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:02.804384 17219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "8dd8a512904b4c7a86c4dd67cc346b54"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "6bcc1e1e7615690db26009aea8b15a8c"
server_key_iv: "551a7f0573398114cfabc25efc769e02"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.805064 17219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "8dd8a512904b4c7a86c4dd67cc346b54"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "6bcc1e1e7615690db26009aea8b15a8c"
server_key_iv: "551a7f0573398114cfabc25efc769e02"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.809857 17219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260621 07:01:02.813114 17234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.814452 17219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:01:02.814582 17219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "8dd8a512904b4c7a86c4dd67cc346b54"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "6bcc1e1e7615690db26009aea8b15a8c"
server_key_iv: "551a7f0573398114cfabc25efc769e02"
server_key_version: "encryptionkey@0"
I20260621 07:01:02.814754 17219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:02.831784 17219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:02.835677 17219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:02.835897 17219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:02.836757 17219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:02.838003 17219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:02.838074 17219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.838157 17219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:02.838201 17219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:02.871780 17219 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37275
I20260621 07:01:02.871871 17346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37275 every 8 connection(s)
I20260621 07:01:02.873077 17219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:01:02.882714 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17219
I20260621 07:01:02.882900 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:01:02.883381 3031 external_mini_cluster.cc:1428] Setting key 41e634345c3f4327984a2384829b70a6
I20260621 07:01:02.884361 17079 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
Jun 21 07:01:02 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025262, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:02.892557 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.875535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:59595 (local address 127.2.245.254:43509)
0621 07:01:02.875727 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.875740 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.876921 (+ 1181us) server_negotiation.cc:407] Connection header received
0621 07:01:02.878590 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.878594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.878673 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.878762 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:02.880244 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.880983 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.882476 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.882915 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.883077 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.887095 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:02.887101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:02.887106 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:02.887147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:02.889625 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.890297 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.890304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.890310 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.890432 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.890877 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.890881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.890885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.891201 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:02.891346 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.892004 (+ 658us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.892232 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:01:02.893507 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.876049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:59595)
0621 07:01:02.876749 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:02.876786 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:02.878350 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:02.879021 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:02.879032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:02.879513 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:02.880090 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.880107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.881113 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.881116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.882291 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.882302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.883166 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.883173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.883411 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.884134 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:02.884160 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:02.886859 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:02.889770 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:02.889779 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:02.889793 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:02.890178 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:02.890549 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:02.890553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:02.890556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:02.890756 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:02.891319 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:02.891326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:02.891657 (+ 331us) client_negotiation.cc:769] Sending connection context
0621 07:01:02.891916 (+ 259us) client_negotiation.cc:240] Negotiation successful
0621 07:01:02.892406 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":286,"threads_started":1}
I20260621 07:01:02.894763 17347 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
I20260621 07:01:02.895069 17347 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:02.895673 17347 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
I20260621 07:01:02.897023 16889 ts_manager.cc:194] Registered new tserver with Master: 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
I20260621 07:01:02.898209 16889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:59595
I20260621 07:01:02.899040 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:01:02.915925 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.904178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.2.245.254:43509)
0621 07:01:02.904409 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.904417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.904826 (+ 409us) server_negotiation.cc:407] Connection header received
0621 07:01:02.904948 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.904952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.905018 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.905095 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:02.906312 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.907093 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.908427 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.908824 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.908910 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.911242 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:02.911247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:02.911250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:02.911276 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:02.913644 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.914227 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.914230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.914233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.914348 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:02.914649 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:02.914652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:02.914654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:02.914997 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:02.915187 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.915415 (+ 228us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.915566 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:01:02.919876 16890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:01:02.926641 16890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:01:02.964089 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.938317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.2.245.194:38153)
0621 07:01:02.953611 (+ 15294us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.953640 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.954871 (+ 1231us) server_negotiation.cc:407] Connection header received
0621 07:01:02.955307 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.955318 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.955621 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.955821 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:02.956756 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.958295 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.961522 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.962491 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.962627 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.962866 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.963612 (+ 746us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.963791 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15096,"thread_start_us":15354,"threads_started":1}
I20260621 07:01:02.964195 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.938132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38153 (local address 127.0.0.1:44006)
0621 07:01:02.954700 (+ 16568us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:02.954743 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:02.954992 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:02.956081 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:02.956088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:02.956120 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:02.956550 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.956565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.958443 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.958448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.961326 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.961344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.963032 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.963038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.963331 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.963401 (+ 70us) client_negotiation.cc:769] Sending connection context
0621 07:01:02.963568 (+ 167us) client_negotiation.cc:240] Negotiation successful
0621 07:01:02.963752 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16041,"mutex_wait_us":100,"thread_start_us":17402,"threads_started":1}
I20260621 07:01:02.964352 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.938696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:56252)
0621 07:01:02.955830 (+ 17134us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:02.955847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:02.956041 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:02.957000 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:02.957003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:02.957024 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:02.957393 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.957408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.959292 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.959297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.961149 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.961161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.963552 (+ 2391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.963558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.963743 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.963771 (+ 28us) client_negotiation.cc:769] Sending connection context
0621 07:01:02.963843 (+ 72us) client_negotiation.cc:240] Negotiation successful
0621 07:01:02.963967 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16993,"mutex_wait_us":123,"thread_start_us":15337,"threads_started":1}
I20260621 07:01:02.964886 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.939065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.2.245.195:37275)
0621 07:01:02.948258 (+ 9193us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.948270 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.955510 (+ 7240us) server_negotiation.cc:407] Connection header received
0621 07:01:02.955908 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.955915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.956128 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.956306 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:02.957337 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.958464 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.961483 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.962774 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.962896 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.963178 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.964407 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.964576 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8992,"thread_start_us":8842,"threads_started":1}
I20260621 07:01:02.964270 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.938374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37275 (local address 127.0.0.1:37768)
0621 07:01:02.955400 (+ 17026us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:02.955419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:02.955638 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:02.956617 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:02.956624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:02.956654 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:02.957087 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.957102 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.958607 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.958613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.961353 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:02.961361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.963340 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.963349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:02.963546 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.963588 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 07:01:02.963677 (+ 89us) client_negotiation.cc:240] Negotiation successful
0621 07:01:02.963800 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16905,"mutex_wait_us":144,"thread_start_us":16736,"threads_started":1}
I20260621 07:01:02.965504 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:02.939433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.2.245.193:36487)
0621 07:01:02.952901 (+ 13468us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:02.952913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:02.955939 (+ 3026us) server_negotiation.cc:407] Connection header received
0621 07:01:02.956220 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:02.956227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:02.956485 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:02.956759 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:02.957584 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.959110 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.961305 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:02.962921 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:02.963085 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:02.963332 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:02.964531 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0621 07:01:02.965269 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13208,"thread_start_us":13004,"threads_started":1}
I20260621 07:01:02.968425 17148 tablet_service.cc:1467] Processing CreateTablet for tablet 25dc3d086a1844ebab9d148e4162b314 (DEFAULT_TABLE table=test-table [id=1c8374dd5f414a3f87f65c7aff887902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:02.968829 17282 tablet_service.cc:1467] Processing CreateTablet for tablet 25dc3d086a1844ebab9d148e4162b314 (DEFAULT_TABLE table=test-table [id=1c8374dd5f414a3f87f65c7aff887902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:02.969141 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 25dc3d086a1844ebab9d148e4162b314 (DEFAULT_TABLE table=test-table [id=1c8374dd5f414a3f87f65c7aff887902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:02.969882 17148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25dc3d086a1844ebab9d148e4162b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:02.969993 17282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25dc3d086a1844ebab9d148e4162b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:02.970425 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25dc3d086a1844ebab9d148e4162b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:02.981412 17365 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Bootstrap starting.
I20260621 07:01:02.984439 17367 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Bootstrap starting.
I20260621 07:01:02.985316 17365 tablet_bootstrap.cc:654] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:02.985597 17366 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Bootstrap starting.
I20260621 07:01:02.986786 17365 log.cc:824] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:02.989269 17366 tablet_bootstrap.cc:654] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:02.989418 17365 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: No bootstrap required, opened a new log
I20260621 07:01:02.989696 17365 ts_tablet_manager.cc:1397] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260621 07:01:02.990543 17366 log.cc:824] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:02.990880 17367 tablet_bootstrap.cc:654] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:02.991974 17367 log.cc:824] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:02.993568 17366 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: No bootstrap required, opened a new log
I20260621 07:01:02.993834 17366 ts_tablet_manager.cc:1397] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260621 07:01:02.994967 17367 tablet_bootstrap.cc:492] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: No bootstrap required, opened a new log
I20260621 07:01:02.995283 17367 ts_tablet_manager.cc:1397] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260621 07:01:02.998216 17365 raft_consensus.cc:348] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:02.998586 17365 raft_consensus.cc:374] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:02.998713 17365 raft_consensus.cc:729] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd8a512904b4c7a86c4dd67cc346b54, State: Initialized, Role: FOLLOWER
I20260621 07:01:02.999336 17365 consensus_queue.cc:260] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.000433 17347 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
I20260621 07:01:03.001089 17365 ts_tablet_manager.cc:1428] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260621 07:01:03.004724 17366 raft_consensus.cc:348] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.005187 17366 raft_consensus.cc:374] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:03.005297 17366 raft_consensus.cc:729] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97361611225943309ca2ae08922d3f81, State: Initialized, Role: FOLLOWER
I20260621 07:01:03.006006 17366 consensus_queue.cc:260] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.006052 17367 raft_consensus.cc:348] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.006451 17367 raft_consensus.cc:374] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:03.006640 17367 raft_consensus.cc:729] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dca01673dd140289249555cd29d9f22, State: Initialized, Role: FOLLOWER
I20260621 07:01:03.007328 17213 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
I20260621 07:01:03.007311 17367 consensus_queue.cc:260] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.007869 17366 ts_tablet_manager.cc:1428] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260621 07:01:03.008994 17367 ts_tablet_manager.cc:1428] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260621 07:01:03.088984 17371 raft_consensus.cc:482] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:03.089318 17371 raft_consensus.cc:504] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.091001 17371 leader_election.cc:288] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:03.096985 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.091541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:53211 (local address 127.2.245.193:36487)
0621 07:01:03.091742 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.091750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.092202 (+ 452us) server_negotiation.cc:407] Connection header received
0621 07:01:03.092337 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.092340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.092400 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.092515 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:03.092883 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.093588 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.095344 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.096105 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.096284 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.096361 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.096614 (+ 253us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.096736 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:01:03.097018 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.091432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.2.245.195:53211)
0621 07:01:03.092022 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:03.092049 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:03.092200 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:03.092537 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:03.092542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:03.092569 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:03.092745 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:03.092753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.093744 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.093750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:03.095191 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:03.095204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.096261 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.096269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:03.096453 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.096477 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:01:03.096588 (+ 111us) client_negotiation.cc:240] Negotiation successful
0621 07:01:03.096716 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":23}
I20260621 07:01:03.097479 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.091573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:59889 (local address 127.2.245.194:38153)
0621 07:01:03.091811 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.091817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.092439 (+ 622us) server_negotiation.cc:407] Connection header received
0621 07:01:03.092623 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.092627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.092684 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.092801 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:03.093345 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.094026 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.095809 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.096584 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.096777 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.096845 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.097113 (+ 268us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.097236 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 07:01:03.097594 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.091612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38153 (local address 127.2.245.195:59889)
0621 07:01:03.092347 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:03.092359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:03.092479 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:03.092834 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:03.092838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:03.092868 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:03.093183 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:03.093197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.094199 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.094204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:03.095651 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:03.095663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.096795 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.096802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:03.096961 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.096984 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 07:01:03.097088 (+ 104us) client_negotiation.cc:240] Negotiation successful
0621 07:01:03.097216 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":1,"thread_start_us":357,"threads_started":1}
I20260621 07:01:03.098093 17034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25dc3d086a1844ebab9d148e4162b314" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dca01673dd140289249555cd29d9f22" is_pre_election: true
I20260621 07:01:03.098709 17034 raft_consensus.cc:2454] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 0.
I20260621 07:01:03.098680 17168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25dc3d086a1844ebab9d148e4162b314" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97361611225943309ca2ae08922d3f81" is_pre_election: true
I20260621 07:01:03.099246 17168 raft_consensus.cc:2454] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 0.
I20260621 07:01:03.099537 17238 leader_election.cc:302] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd8a512904b4c7a86c4dd67cc346b54, 9dca01673dd140289249555cd29d9f22; no voters:
I20260621 07:01:03.099982 17371 raft_consensus.cc:2781] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:01:03.100138 17371 raft_consensus.cc:482] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:01:03.100248 17371 raft_consensus.cc:3037] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:03.102763 17371 raft_consensus.cc:504] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.103374 17371 leader_election.cc:288] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 election: Requested vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:03.103915 17168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25dc3d086a1844ebab9d148e4162b314" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97361611225943309ca2ae08922d3f81"
I20260621 07:01:03.103931 17034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25dc3d086a1844ebab9d148e4162b314" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dca01673dd140289249555cd29d9f22"
I20260621 07:01:03.104108 17168 raft_consensus.cc:3037] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:03.104135 17034 raft_consensus.cc:3037] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Advancing to term 1
W20260621 07:01:03.104480 17080 tablet.cc:2369] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:01:03.106294 17168 raft_consensus.cc:2454] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 1.
I20260621 07:01:03.106709 17236 leader_election.cc:302] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81; no voters:
I20260621 07:01:03.106750 17034 raft_consensus.cc:2454] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 1.
I20260621 07:01:03.107199 17371 raft_consensus.cc:2781] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:03.107695 17371 raft_consensus.cc:686] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 LEADER]: Becoming Leader. State: Replica: 8dd8a512904b4c7a86c4dd67cc346b54, State: Running, Role: LEADER
I20260621 07:01:03.108239 17371 consensus_queue.cc:237] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:03.112972 16890 catalog_manager.cc:5495] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } health_report { overall_health: UNKNOWN } } }
W20260621 07:01:03.124575 17348 tablet.cc:2369] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:01:03.136754 17214 tablet.cc:2369] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:01:03.156807 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.151517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.2.245.195:37275)
0621 07:01:03.151754 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.151760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.151853 (+ 93us) server_negotiation.cc:407] Connection header received
0621 07:01:03.152094 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.152098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.152180 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.152330 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:03.152660 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.153435 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.154843 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.155291 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.155384 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.155836 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:03.156338 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:01:03.156448 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.156530 (+ 82us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.156595 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:01:03.164820 17168 raft_consensus.cc:1261] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Refusing update from remote peer 8dd8a512904b4c7a86c4dd67cc346b54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:03.164829 17034 raft_consensus.cc:1261] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Refusing update from remote peer 8dd8a512904b4c7a86c4dd67cc346b54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:03.165812 17375 consensus_queue.cc:1035] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:03.166139 17371 consensus_queue.cc:1035] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:03.189846 16889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:03.190088 16889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:03.193491 16889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Sending DeleteTablet for 3 replicas of tablet 25dc3d086a1844ebab9d148e4162b314
I20260621 07:01:03.194554 17148 tablet_service.cc:1514] Processing DeleteTablet for tablet 25dc3d086a1844ebab9d148e4162b314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:03 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:44006
I20260621 07:01:03.194736 17282 tablet_service.cc:1514] Processing DeleteTablet for tablet 25dc3d086a1844ebab9d148e4162b314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:03 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37768
I20260621 07:01:03.195003 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet 25dc3d086a1844ebab9d148e4162b314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:03 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:56252
I20260621 07:01:03.195839 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16951
I20260621 07:01:03.207360 17387 tablet_replica.cc:330] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: stopping tablet replica
I20260621 07:01:03.207901 17389 tablet_replica.cc:330] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: stopping tablet replica
I20260621 07:01:03.208323 17388 tablet_replica.cc:330] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: stopping tablet replica
I20260621 07:01:03.208465 17389 raft_consensus.cc:2229] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:03.208786 17388 raft_consensus.cc:2229] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:03.209008 17387 raft_consensus.cc:2229] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:01:03.209357 17388 raft_consensus.cc:2258] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:03.210009 17387 raft_consensus.cc:2258] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:03.210290 17389 raft_consensus.cc:2258] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:03.211478 17388 ts_tablet_manager.cc:1905] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:03.215189 17389 ts_tablet_manager.cc:1905] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:03.216514 17387 ts_tablet_manager.cc:1905] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:03.217447 17388 ts_tablet_manager.cc:1918] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:03.217773 17388 log.cc:1196] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/25dc3d086a1844ebab9d148e4162b314
I20260621 07:01:03.218353 17388 ts_tablet_manager.cc:1939] T 25dc3d086a1844ebab9d148e4162b314 P 97361611225943309ca2ae08922d3f81: Deleting consensus metadata
I20260621 07:01:03.223342 16875 catalog_manager.cc:4841] TS 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): tablet 25dc3d086a1844ebab9d148e4162b314 (table test-table [id=1c8374dd5f414a3f87f65c7aff887902]) successfully deleted
Jun 21 07:01:03 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:01:03.229323 17387 ts_tablet_manager.cc:1918] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:03.229573 17387 log.cc:1196] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/25dc3d086a1844ebab9d148e4162b314
I20260621 07:01:03.230129 17389 ts_tablet_manager.cc:1918] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:03.230190 17387 ts_tablet_manager.cc:1939] T 25dc3d086a1844ebab9d148e4162b314 P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting consensus metadata
I20260621 07:01:03.230358 17389 log.cc:1196] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/25dc3d086a1844ebab9d148e4162b314
I20260621 07:01:03.230968 17389 ts_tablet_manager.cc:1939] T 25dc3d086a1844ebab9d148e4162b314 P 9dca01673dd140289249555cd29d9f22: Deleting consensus metadata
I20260621 07:01:03.232161 16876 catalog_manager.cc:4841] TS 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): tablet 25dc3d086a1844ebab9d148e4162b314 (table test-table [id=1c8374dd5f414a3f87f65c7aff887902]) successfully deleted
I20260621 07:01:03.232658 16877 catalog_manager.cc:4841] TS 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487): tablet 25dc3d086a1844ebab9d148e4162b314 (table test-table [id=1c8374dd5f414a3f87f65c7aff887902]) successfully deleted
I20260621 07:01:03.234544 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.197163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.2.245.193:36487)
0621 07:01:03.197381 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.197390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.207662 (+ 10272us) server_negotiation.cc:407] Connection header received
0621 07:01:03.207872 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.207879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.207984 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.208089 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:03.210446 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.211686 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.213956 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.214617 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.214756 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.226775 (+ 12019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:03.226807 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:03.226825 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:03.226867 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:03.231117 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.232087 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.232095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.232098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.232291 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.232777 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.232784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.232788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.233179 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:03.233460 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.233827 (+ 367us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.234034 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:01:03.235175 17074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:03.325068 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16951
I20260621 07:01:03.343111 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17085
Jun 21 07:01:03 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:01:03.359544 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.343963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.2.245.194:38153)
0621 07:01:03.344135 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.344143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.344165 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:01:03.344380 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.344389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.344463 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.344570 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:03.345904 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.346628 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.348117 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.348510 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.348647 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.353052 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:03.353072 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:03.353093 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:03.353135 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:03.356756 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.357406 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.357416 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.357419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.357567 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.357941 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.357947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.357950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.358426 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:03.358715 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.358960 (+ 245us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.359178 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:01:03.360311 17208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:03.449296 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17085
I20260621 07:01:03.466421 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17219
Jun 21 07:01:03 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:01:03.483783 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:03.467338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.2.245.195:37275)
0621 07:01:03.467534 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:03.467543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:03.467639 (+ 96us) server_negotiation.cc:407] Connection header received
0621 07:01:03.467868 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:03.467873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:03.467951 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:03.468064 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:03.469434 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.470267 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.471660 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:03.472139 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:03.472312 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:03.476598 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:03.476612 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:03.476632 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:03.476682 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:03.480729 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.481363 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.481373 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.481377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.481596 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:03.481999 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:03.482005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:03.482010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:03.482486 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:03.482822 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:03.483007 (+ 185us) server_negotiation.cc:299] Negotiation successful
0621 07:01:03.483263 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260621 07:01:03.484575 17342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:03.606616 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17219
I20260621 07:01:03.622639 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 16858
I20260621 07:01:03.623883 16920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:03.735497 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 16858
I20260621 07:01:03.756577 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--webserver_interface=127.2.245.254
--webserver_port=41669
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:04.073951 17395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:04.074193 17395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:04.074360 17395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:04.081284 17395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:01:04.081379 17395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:04.081502 17395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:04.081602 17395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:01:04.081683 17395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:01:04.091156 17395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=41669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:04.092314 17395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:04.093565 17395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:04.115031 17402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:04.115083 17404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:04.115113 17401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:04.116247 17395 server_base.cc:1034] running on GCE node
I20260621 07:01:04.117019 17395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:04.118485 17395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:04.119694 17395 hybrid_clock.cc:648] HybridClock initialized: now 1782025264119621 us; error 56 us; skew 500 ppm
I20260621 07:01:04.120108 17395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:04 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:04.123498 17395 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:01:04.125114 17395 webserver.cc:460] Webserver started at http://127.2.245.254:41669/ using document root <none> and password file <none>
I20260621 07:01:04.125857 17395 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:04.126017 17395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:04.131883 17395 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260621 07:01:04.134788 17411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:04.136133 17395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:04.136343 17395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "631dc01aaf9d4e9b819863428136d264"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "5c1113167e9dc96c924e9825e0e69821"
server_key_iv: "71eec2470d8c64c691df7f5c87057e90"
server_key_version: "encryptionkey@0"
I20260621 07:01:04.137439 17395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:04.166817 17395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:04.170956 17395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:04.171248 17395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:04.190189 17395 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43509
I20260621 07:01:04.190199 17462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43509 every 8 connection(s)
I20260621 07:01:04.191867 17395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:01:04.197120 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17395
I20260621 07:01:04.197930 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:36487
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=39059
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:01:04.198808 17463 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:01:04.202118 17463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap starting.
I20260621 07:01:04.229046 17463 log.cc:824] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:04.244609 17463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:01:04.245246 17463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap complete.
I20260621 07:01:04.258759 17463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:04.260046 17463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Initialized, Role: FOLLOWER
I20260621 07:01:04.260758 17463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:04.261065 17463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:01:04.261199 17463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:01:04.261369 17463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:01:04.267767 17463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:04.268473 17463 leader_election.cc:302] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631dc01aaf9d4e9b819863428136d264; no voters:
I20260621 07:01:04.269093 17463 leader_election.cc:288] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:01:04.269238 17467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:01:04.270838 17467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 LEADER]: Becoming Leader. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Running, Role: LEADER
I20260621 07:01:04.271432 17467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:04.271802 17463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:01:04.274021 17468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:04.273952 17469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631dc01aaf9d4e9b819863428136d264. Latest consensus state: current_term: 2 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:04.274307 17469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:04.274304 17468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:04.288246 17473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:01:04.296160 17473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:01:04.297648 17473 catalog_manager.cc:1259] Loaded cluster ID: 620def677f10468e917425330d473a14
I20260621 07:01:04.297811 17473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:01:04.301122 17473 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260621 07:01:04.315836 17484 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:01:04.319566 17473 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260621 07:01:04.328615 17473 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:01:04.330394 17473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Loaded TSK: 0
I20260621 07:01:04.331969 17473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:01:04.571326 17465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:04.571683 17465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:04.571853 17465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:04.578874 17465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:04.579043 17465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:04.579209 17465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:01:04.588770 17465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:36487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=39059
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:04.590209 17465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:04.591571 17465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:04.605646 17490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:04.605669 17491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:04.606392 17465 server_base.cc:1034] running on GCE node
W20260621 07:01:04.606448 17493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:04.607122 17465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:04.608053 17465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:04.609280 17465 hybrid_clock.cc:648] HybridClock initialized: now 1782025264609225 us; error 48 us; skew 500 ppm
I20260621 07:01:04.609586 17465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:04 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:04.612804 17465 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:01:04.614368 17465 webserver.cc:460] Webserver started at http://127.2.245.193:39059/ using document root <none> and password file <none>
I20260621 07:01:04.615156 17465 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:04.615329 17465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:04.620738 17465 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260621 07:01:04.623554 17500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:04.625007 17465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 07:01:04.625185 17465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "9dca01673dd140289249555cd29d9f22"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "2c3087147a9239f6cb84b4859b3802cc"
server_key_iv: "bc73fc54a460d437436e56f87d6af35d"
server_key_version: "encryptionkey@0"
I20260621 07:01:04.626132 17465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:04.656947 17465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:04.660925 17465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:04.661204 17465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:04.662150 17465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:04.663492 17465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:04.663609 17465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:04.663740 17465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:04.663841 17465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:04.697330 17465 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36487
I20260621 07:01:04.697420 17612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36487 every 8 connection(s)
I20260621 07:01:04.698812 17465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:01:04.706429 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17465
I20260621 07:01:04.707111 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:38153
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=33477
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:01:04 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:04.724443 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:04.701940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:42963)
0621 07:01:04.702733 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:04.702777 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:04.704484 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:04.705887 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:04.705899 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:04.706512 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:04.707145 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:04.707170 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:04.708840 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:04.708849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:04.710934 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:04.710978 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:04.712013 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:04.712023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:04.712375 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:04.713089 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:04.713123 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:04.715766 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:04.720358 (+ 4592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:04.720365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:04.720376 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:04.720843 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:04.721390 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:04.721393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:04.721395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:04.721562 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:04.722534 (+ 972us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:04.722539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:04.722850 (+ 311us) client_negotiation.cc:769] Sending connection context
0621 07:01:04.723149 (+ 299us) client_negotiation.cc:240] Negotiation successful
0621 07:01:04.723507 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":308,"threads_started":1}
I20260621 07:01:04.724977 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:04.701783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:42963 (local address 127.2.245.254:43509)
0621 07:01:04.702476 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:04.702491 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:04.702906 (+ 415us) server_negotiation.cc:407] Connection header received
0621 07:01:04.705019 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:04.705030 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:04.705341 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:04.705754 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:04.707433 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:04.708649 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:04.711108 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:04.711753 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:04.711913 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:04.716058 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:04.716070 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:04.716090 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:04.716139 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:04.720194 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:04.721019 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:04.721030 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:04.721040 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:04.721270 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:04.721710 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:04.721718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:04.721724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:04.722402 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:04.722610 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:04.723410 (+ 800us) server_negotiation.cc:299] Negotiation successful
0621 07:01:04.723815 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":226,"threads_started":1}
I20260621 07:01:04.727728 17613 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
I20260621 07:01:04.728070 17613 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:04.729035 17613 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
I20260621 07:01:04.731648 17428 ts_manager.cc:194] Registered new tserver with Master: 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:04.733137 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:42963
W20260621 07:01:05.037815 17617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:05.038097 17617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:05.038249 17617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:05.045908 17617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:05.046010 17617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:05.046180 17617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:01:05.056041 17617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:38153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=33477
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:05.057317 17617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:05.058642 17617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:05.071895 17625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:05.071831 17623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:05.071837 17622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:05.073042 17617 server_base.cc:1034] running on GCE node
I20260621 07:01:05.073601 17617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:05.074498 17617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:05.075712 17617 hybrid_clock.cc:648] HybridClock initialized: now 1782025265075662 us; error 36 us; skew 500 ppm
I20260621 07:01:05.075986 17617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:05 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:05.079046 17617 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:01:05.080413 17617 webserver.cc:460] Webserver started at http://127.2.245.194:33477/ using document root <none> and password file <none>
I20260621 07:01:05.081048 17617 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:05.081161 17617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:05.086586 17617 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:01:05.089516 17632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:05.090782 17617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:01:05.090911 17617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "97361611225943309ca2ae08922d3f81"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "511269161b76b6416371522bccb0ae8d"
server_key_iv: "6aeff80d1dc5c22b0bb25a0c5b25d543"
server_key_version: "encryptionkey@0"
I20260621 07:01:05.091912 17617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:05.115869 17617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:05.120185 17617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:05.120404 17617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:05.121330 17617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:05.122639 17617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:05.122802 17617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:05.122910 17617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:05.123060 17617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260621 07:01:05.155669 17617 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38153
I20260621 07:01:05.155766 17744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38153 every 8 connection(s)
I20260621 07:01:05.157115 17617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:01:05.163973 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17617
I20260621 07:01:05.164609 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:37275
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=34663
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:01:05 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:05.176618 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.159581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:56903 (local address 127.2.245.254:43509)
0621 07:01:05.159782 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.159786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.160854 (+ 1068us) server_negotiation.cc:407] Connection header received
0621 07:01:05.162680 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.162687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.162777 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.162883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:05.164438 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.165272 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.166822 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.167437 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.167575 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.170859 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:05.170865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:05.170876 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:05.170908 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:05.173550 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.174255 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.174262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.174268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.174402 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.174817 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.174822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.174826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.175167 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:05.175335 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.176027 (+ 692us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.176255 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:01:05.177577 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.160013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:56903)
0621 07:01:05.160684 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.160728 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.162413 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.163129 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.163137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.163631 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:05.164251 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.164289 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.165434 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.165438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.166639 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.166652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.167663 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.167670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.167918 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.168567 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:05.168588 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:05.170644 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:05.173686 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:05.173695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:05.173710 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:05.174135 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:05.174511 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:05.174517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:05.174520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:05.174704 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:05.175310 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:05.175317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:05.175644 (+ 327us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.175923 (+ 279us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.176457 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":263,"threads_started":1}
I20260621 07:01:05.179116 17745 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
I20260621 07:01:05.179450 17745 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:05.180331 17745 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
I20260621 07:01:05.182000 17428 ts_manager.cc:194] Registered new tserver with Master: 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153)
I20260621 07:01:05.183348 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:56903
W20260621 07:01:05.494225 17748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:05.494494 17748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:05.494582 17748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:05.501976 17748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:05.502081 17748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:05.502192 17748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:01:05.512593 17748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:37275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=34663
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:05.513916 17748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:05.515276 17748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:05.528715 17753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:05.528688 17754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:05.528781 17756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:05.529452 17748 server_base.cc:1034] running on GCE node
I20260621 07:01:05.530104 17748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:05.531102 17748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:05.532311 17748 hybrid_clock.cc:648] HybridClock initialized: now 1782025265532265 us; error 40 us; skew 500 ppm
I20260621 07:01:05.532632 17748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:05 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:05.535727 17748 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:01:05.537119 17748 webserver.cc:460] Webserver started at http://127.2.245.195:34663/ using document root <none> and password file <none>
I20260621 07:01:05.537739 17748 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:05.537847 17748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:05.543185 17748 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:01:05.545693 17763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:05.547232 17748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:01:05.547403 17748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "8dd8a512904b4c7a86c4dd67cc346b54"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "6bcc1e1e7615690db26009aea8b15a8c"
server_key_iv: "551a7f0573398114cfabc25efc769e02"
server_key_version: "encryptionkey@0"
I20260621 07:01:05.548333 17748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:05.567184 17748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:05.571327 17748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:05.571621 17748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:05.572654 17748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:05.574074 17748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:05.574203 17748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:05.574339 17748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:05.574424 17748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:05.608135 17748 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37275
I20260621 07:01:05.608227 17875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37275 every 8 connection(s)
I20260621 07:01:05.609499 17748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:01:05.618326 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17748
Jun 21 07:01:05 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:05.631033 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.612153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:53119 (local address 127.2.245.254:43509)
0621 07:01:05.612362 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.612368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.613487 (+ 1119us) server_negotiation.cc:407] Connection header received
0621 07:01:05.615233 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.615237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.615321 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.615408 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:05.616845 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.617553 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.619408 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.620181 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.620331 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.624136 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:05.624143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:05.624147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:05.624183 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:05.627566 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.628276 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.628282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.628285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.628441 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.628933 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.628936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.628938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.629299 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:05.629446 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.630415 (+ 969us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.630651 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:01:05.632099 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.612645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:53119)
0621 07:01:05.613330 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.613366 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.614936 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.615603 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.615611 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.616122 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:05.616653 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.616670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.617730 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.617735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.619247 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.619272 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.620423 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.620430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.620780 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.621467 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:05.621489 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:05.623928 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:05.627719 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:05.627726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:05.627738 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:05.628141 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:05.628575 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:05.628580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:05.628583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:05.628815 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:05.629450 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:05.629458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:05.629955 (+ 497us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.630266 (+ 311us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.630775 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":267,"threads_started":1}
I20260621 07:01:05.633605 17876 heartbeater.cc:347] Connected to a master server at 127.2.245.254:43509
I20260621 07:01:05.633886 17876 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:05.634585 17876 heartbeater.cc:511] Master 127.2.245.254:43509 requested a full tablet report, sending...
I20260621 07:01:05.634794 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.619242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.2.245.254:43509)
0621 07:01:05.619945 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.619951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.620175 (+ 224us) server_negotiation.cc:407] Connection header received
0621 07:01:05.620353 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.620361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.620437 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.620554 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:05.622182 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.623267 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.625005 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.625485 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.625582 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.628636 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:05.628645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:05.628651 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:05.628684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:05.632006 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.632794 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.632800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.632807 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.632913 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.633367 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.633371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.633373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.633602 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:05.633852 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.634166 (+ 314us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.634390 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":277,"threads_started":1}
I20260621 07:01:05.636082 17428 ts_manager.cc:194] Registered new tserver with Master: 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
I20260621 07:01:05.637039 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:53119
I20260621 07:01:05.637310 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:01:05.644177 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.638244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.2.245.254:43509)
0621 07:01:05.638429 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.638436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.638884 (+ 448us) server_negotiation.cc:407] Connection header received
0621 07:01:05.639134 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.639140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.639282 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.639428 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:05.639862 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.640602 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.642110 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.642552 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.642681 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.643226 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:05.643650 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:01:05.643826 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.643898 (+ 72us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.643966 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:01:05.644932 17428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:01:05.652081 17428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:01:05.674398 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.663359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38153 (local address 127.0.0.1:34408)
0621 07:01:05.664357 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.664403 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.664590 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.666341 (+ 1751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.666348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.666374 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:05.666847 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.666860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.669140 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.669145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.671552 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.671568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.673480 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.673486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.673679 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.673747 (+ 68us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.673852 (+ 105us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.674025 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":1414,"threads_started":1}
I20260621 07:01:05.675145 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.664306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.2.245.195:37275)
0621 07:01:05.665806 (+ 1500us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.665813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.666450 (+ 637us) server_negotiation.cc:407] Connection header received
0621 07:01:05.666553 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.666562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.666804 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.667007 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:05.667891 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.669407 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.671790 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.673255 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.673384 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.673610 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.674635 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.674804 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1331,"thread_start_us":1436,"threads_started":1}
I20260621 07:01:05.674515 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.663435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37275 (local address 127.0.0.1:55516)
0621 07:01:05.666215 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.666240 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.666343 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.667274 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.667278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.667301 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:05.667682 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.667697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.669577 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.669582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.671623 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.671644 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.673767 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.673773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.673943 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.673972 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.674054 (+ 82us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.674170 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2659,"mutex_wait_us":1,"thread_start_us":2553,"threads_started":1}
I20260621 07:01:05.676054 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.663481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.2.245.194:38153)
0621 07:01:05.665695 (+ 2214us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.665703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.665735 (+ 32us) server_negotiation.cc:407] Connection header received
0621 07:01:05.665830 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.665839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.666103 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.667076 (+ 973us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:05.667358 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.668977 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.671780 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.673261 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.675252 (+ 1991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.675479 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.675554 (+ 75us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.675716 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2056,"thread_start_us":1946,"threads_started":1}
I20260621 07:01:05.679777 17811 tablet_service.cc:1467] Processing CreateTablet for tablet d8f85e68553b440a8de37e60fe117bae (DEFAULT_TABLE table=test-table [id=b31fff57c8904f2093094da3f70c2609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:05.679772 17680 tablet_service.cc:1467] Processing CreateTablet for tablet d8f85e68553b440a8de37e60fe117bae (DEFAULT_TABLE table=test-table [id=b31fff57c8904f2093094da3f70c2609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:05.681258 17680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f85e68553b440a8de37e60fe117bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:05.681273 17811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f85e68553b440a8de37e60fe117bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:05.683586 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.665215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37076)
0621 07:01:05.674899 (+ 9684us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.674918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.675029 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.678055 (+ 3026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.678060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.678083 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:05.678236 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.678247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.679880 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.679888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.681548 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.681560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.682828 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.682833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.683036 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.683078 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.683188 (+ 110us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.683324 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9594,"thread_start_us":13063,"threads_started":1}
I20260621 07:01:05.683687 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.665581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.2.245.193:36487)
0621 07:01:05.677604 (+ 12023us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.677610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.677632 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:01:05.677703 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.677710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.677867 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.678072 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:05.678408 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.679681 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.681698 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.682643 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.682809 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.683031 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.683234 (+ 203us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.683384 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11887,"thread_start_us":13161,"threads_started":1}
I20260621 07:01:05.686904 17548 tablet_service.cc:1467] Processing CreateTablet for tablet d8f85e68553b440a8de37e60fe117bae (DEFAULT_TABLE table=test-table [id=b31fff57c8904f2093094da3f70c2609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:05.688321 17548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f85e68553b440a8de37e60fe117bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:05.710395 17893 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Bootstrap starting.
I20260621 07:01:05.715718 17892 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Bootstrap starting.
I20260621 07:01:05.717381 17893 tablet_bootstrap.cc:654] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:05.717621 17894 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Bootstrap starting.
I20260621 07:01:05.719003 17893 log.cc:824] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:05.720062 17892 tablet_bootstrap.cc:654] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:05.721192 17894 tablet_bootstrap.cc:654] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:05.721371 17892 log.cc:824] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:05.721931 17893 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: No bootstrap required, opened a new log
I20260621 07:01:05.722285 17893 ts_tablet_manager.cc:1397] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260621 07:01:05.722420 17894 log.cc:824] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:05.724601 17892 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: No bootstrap required, opened a new log
I20260621 07:01:05.724879 17892 ts_tablet_manager.cc:1397] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260621 07:01:05.725479 17894 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: No bootstrap required, opened a new log
I20260621 07:01:05.725734 17894 ts_tablet_manager.cc:1397] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260621 07:01:05.730835 17893 raft_consensus.cc:348] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.731300 17893 raft_consensus.cc:374] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:05.731437 17893 raft_consensus.cc:729] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dca01673dd140289249555cd29d9f22, State: Initialized, Role: FOLLOWER
I20260621 07:01:05.732052 17893 consensus_queue.cc:260] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.733146 17613 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
I20260621 07:01:05.733855 17893 ts_tablet_manager.cc:1428] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260621 07:01:05.735728 17892 raft_consensus.cc:348] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.736176 17892 raft_consensus.cc:374] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:05.736299 17892 raft_consensus.cc:729] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd8a512904b4c7a86c4dd67cc346b54, State: Initialized, Role: FOLLOWER
I20260621 07:01:05.736239 17894 raft_consensus.cc:348] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.736708 17894 raft_consensus.cc:374] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:05.736822 17894 raft_consensus.cc:729] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97361611225943309ca2ae08922d3f81, State: Initialized, Role: FOLLOWER
I20260621 07:01:05.736894 17892 consensus_queue.cc:260] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.737380 17894 consensus_queue.cc:260] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.738061 17876 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
I20260621 07:01:05.738718 17892 ts_tablet_manager.cc:1428] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260621 07:01:05.739666 17745 heartbeater.cc:503] Master 127.2.245.254:43509 was elected leader, sending a full tablet report...
I20260621 07:01:05.740371 17894 ts_tablet_manager.cc:1428] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260621 07:01:05.744242 17899 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:05.744432 17899 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.745859 17899 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:05.752472 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.746214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:56425 (local address 127.2.245.194:38153)
0621 07:01:05.746400 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.746405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.747077 (+ 672us) server_negotiation.cc:407] Connection header received
0621 07:01:05.747266 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.747274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.747353 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.747417 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:05.747837 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.748635 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.750571 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.751461 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.751633 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.751731 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.752068 (+ 337us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.752218 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:01:05.752475 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.746290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38153 (local address 127.2.245.195:56425)
0621 07:01:05.746932 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.746992 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.747102 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.747466 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.747471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.747493 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:05.747691 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.747709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.748814 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.748819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.750425 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.750437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.751637 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.751645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.751886 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.751939 (+ 53us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.752060 (+ 121us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.752193 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515}
I20260621 07:01:05.752557 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.746715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:53011 (local address 127.2.245.193:36487)
0621 07:01:05.746964 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.746971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.747153 (+ 182us) server_negotiation.cc:407] Connection header received
0621 07:01:05.747319 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.747322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.747374 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.747478 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:05.748131 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.748942 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.750723 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.751595 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.751746 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.751821 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.752198 (+ 377us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.752326 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260621 07:01:05.752712 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.746325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.2.245.195:53011)
0621 07:01:05.747024 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:05.747059 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:05.747210 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:05.747512 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:05.747516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:05.747557 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:05.747955 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.747967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.749101 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.749105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.750565 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:05.750579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.751795 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.751799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:05.751971 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.752025 (+ 54us) client_negotiation.cc:769] Sending connection context
0621 07:01:05.752159 (+ 134us) client_negotiation.cc:240] Negotiation successful
0621 07:01:05.752276 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":1,"thread_start_us":324,"threads_started":1}
I20260621 07:01:05.753710 17700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f85e68553b440a8de37e60fe117bae" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97361611225943309ca2ae08922d3f81" is_pre_election: true
I20260621 07:01:05.753706 17568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f85e68553b440a8de37e60fe117bae" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dca01673dd140289249555cd29d9f22" is_pre_election: true
I20260621 07:01:05.754181 17700 raft_consensus.cc:2454] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 0.
I20260621 07:01:05.754194 17568 raft_consensus.cc:2454] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 0.
I20260621 07:01:05.754812 17767 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd8a512904b4c7a86c4dd67cc346b54, 9dca01673dd140289249555cd29d9f22; no voters:
I20260621 07:01:05.755260 17899 raft_consensus.cc:2781] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:01:05.755373 17899 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:01:05.755441 17899 raft_consensus.cc:3037] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:05.757556 17899 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.758140 17899 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 election: Requested vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487)
I20260621 07:01:05.758471 17700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f85e68553b440a8de37e60fe117bae" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97361611225943309ca2ae08922d3f81"
I20260621 07:01:05.758653 17700 raft_consensus.cc:3037] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:05.758646 17568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f85e68553b440a8de37e60fe117bae" candidate_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dca01673dd140289249555cd29d9f22"
I20260621 07:01:05.758845 17568 raft_consensus.cc:3037] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:05.760887 17700 raft_consensus.cc:2454] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 1.
I20260621 07:01:05.761016 17568 raft_consensus.cc:2454] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd8a512904b4c7a86c4dd67cc346b54 in term 1.
I20260621 07:01:05.761350 17765 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81; no voters:
I20260621 07:01:05.761685 17899 raft_consensus.cc:2781] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:05.762216 17899 raft_consensus.cc:686] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 LEADER]: Becoming Leader. State: Replica: 8dd8a512904b4c7a86c4dd67cc346b54, State: Running, Role: LEADER
I20260621 07:01:05.762773 17899 consensus_queue.cc:237] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:05.767145 17428 catalog_manager.cc:5495] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } health_report { overall_health: UNKNOWN } } }
I20260621 07:01:05.799239 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.793518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.2.245.195:37275)
0621 07:01:05.793737 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.793756 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.793783 (+ 27us) server_negotiation.cc:407] Connection header received
0621 07:01:05.794053 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.794060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.794141 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.794301 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:05.794685 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.795620 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.797098 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.797506 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.797648 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.798159 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:05.798640 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:01:05.798822 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.798871 (+ 49us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.798974 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260621 07:01:05.807871 17568 raft_consensus.cc:1261] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Refusing update from remote peer 8dd8a512904b4c7a86c4dd67cc346b54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:05.807870 17700 raft_consensus.cc:1261] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Refusing update from remote peer 8dd8a512904b4c7a86c4dd67cc346b54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:05.808996 17902 consensus_queue.cc:1035] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:05.809386 17899 consensus_queue.cc:1035] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:05.821638 17904 mvcc.cc:205] Tried to move back new op lower bound from 7299175488737611776 to 7299175488568655872. Current Snapshot: MvccSnapshot[applied={T|T < 7299175488737611776}]
I20260621 07:01:05.823349 17906 mvcc.cc:205] Tried to move back new op lower bound from 7299175488737611776 to 7299175488568655872. Current Snapshot: MvccSnapshot[applied={T|T < 7299175488737611776}]
I20260621 07:01:05.835078 17427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:05.835391 17427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:05.838860 17427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Sending DeleteTablet for 3 replicas of tablet d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:05.840034 17811 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f85e68553b440a8de37e60fe117bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:05 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:55516
I20260621 07:01:05.840047 17680 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f85e68553b440a8de37e60fe117bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:05 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:34408
I20260621 07:01:05.840111 17548 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f85e68553b440a8de37e60fe117bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:05 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:37076
I20260621 07:01:05.842470 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17465
I20260621 07:01:05.855474 17914 tablet_replica.cc:330] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: stopping tablet replica
I20260621 07:01:05.855752 17916 tablet_replica.cc:330] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: stopping tablet replica
I20260621 07:01:05.854894 17915 tablet_replica.cc:330] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: stopping tablet replica
I20260621 07:01:05.856364 17915 raft_consensus.cc:2229] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:05.856462 17916 raft_consensus.cc:2229] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:05.856755 17914 raft_consensus.cc:2229] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:01:05.857287 17916 raft_consensus.cc:2258] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:05.857306 17915 raft_consensus.cc:2258] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:05.858562 17914 raft_consensus.cc:2258] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:05.859759 17916 ts_tablet_manager.cc:1905] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:05.860476 17914 ts_tablet_manager.cc:1905] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:05.862298 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:05.843490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.2.245.193:36487)
0621 07:01:05.843658 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:05.843664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:05.843764 (+ 100us) server_negotiation.cc:407] Connection header received
0621 07:01:05.843971 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:05.843976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:05.844070 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:05.844170 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:05.846446 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.847728 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.850144 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:05.850840 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:05.850977 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:05.855130 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:05.855146 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:05.855165 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:05.855224 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:05.859055 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.859908 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.859917 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.859920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.860101 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:05.860569 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:05.860577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:05.860583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:05.861067 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:05.861365 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:05.861696 (+ 331us) server_negotiation.cc:299] Negotiation successful
0621 07:01:05.861915 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:01:05.862728 17915 ts_tablet_manager.cc:1905] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:05.863230 17608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:05.867231 17916 ts_tablet_manager.cc:1918] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:05.867463 17916 log.cc:1196] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:05.868113 17916 ts_tablet_manager.cc:1939] T d8f85e68553b440a8de37e60fe117bae P 97361611225943309ca2ae08922d3f81: Deleting consensus metadata
I20260621 07:01:05.870159 17413 catalog_manager.cc:4841] TS 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): tablet d8f85e68553b440a8de37e60fe117bae (table test-table [id=b31fff57c8904f2093094da3f70c2609]) successfully deleted
I20260621 07:01:05.871613 17914 ts_tablet_manager.cc:1918] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:05.872025 17914 log.cc:1196] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:05.872848 17914 ts_tablet_manager.cc:1939] T d8f85e68553b440a8de37e60fe117bae P 8dd8a512904b4c7a86c4dd67cc346b54: Deleting consensus metadata
I20260621 07:01:05.874893 17414 catalog_manager.cc:4841] TS 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): tablet d8f85e68553b440a8de37e60fe117bae (table test-table [id=b31fff57c8904f2093094da3f70c2609]) successfully deleted
I20260621 07:01:05.995615 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17465
W20260621 07:01:06.018785 17415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:36487 (error 108)
I20260621 07:01:06.019553 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17617
I20260621 07:01:06.021099 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.020624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37098)
0621 07:01:06.020834 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.020937 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260621 07:01:06.021381 17415 catalog_manager.cc:4584] TS 9dca01673dd140289249555cd29d9f22 (127.2.245.193:36487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8f85e68553b440a8de37e60fe117bae: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
I20260621 07:01:06.041071 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.020507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.2.245.194:38153)
0621 07:01:06.020694 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:06.020700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:06.021262 (+ 562us) server_negotiation.cc:407] Connection header received
0621 07:01:06.021343 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:06.021348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:06.021433 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:06.021564 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:06.023273 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:06.024002 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:06.025934 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:06.026343 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:06.026436 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:06.035139 (+ 8703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:06.035157 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:06.035175 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:06.035232 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:06.038080 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:06.039048 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:06.039055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:06.039057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:06.039173 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:06.039587 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:06.039590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:06.039592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:06.039968 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:06.040177 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:06.040513 (+ 336us) server_negotiation.cc:299] Negotiation successful
0621 07:01:06.040668 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260621 07:01:06.041927 17740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:06.075328 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.074794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37102)
0621 07:01:06.075049 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.075157 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260621 07:01:06.076539 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.076081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37116)
0621 07:01:06.076258 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.076361 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:01:06.132295 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17617
I20260621 07:01:06.136842 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.136429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37130)
0621 07:01:06.136623 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.136705 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260621 07:01:06.137920 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.137494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37132)
0621 07:01:06.137634 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.137708 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260621 07:01:06.155606 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17748
I20260621 07:01:06.176998 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.156470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.2.245.195:37275)
0621 07:01:06.156652 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:06.156657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:06.156676 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:01:06.156899 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:06.156903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:06.156967 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:06.157076 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:06.158921 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:06.159930 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:06.161843 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:06.162381 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:06.162487 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:06.170490 (+ 8003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:06.170501 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:06.170516 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:06.170563 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:06.173912 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:06.174893 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:06.174900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:06.174902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:06.175123 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:06.175542 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:06.175545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:06.175547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:06.175878 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:06.176123 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:06.176495 (+ 372us) server_negotiation.cc:299] Negotiation successful
0621 07:01:06.176656 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:01:06.177975 17871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:06.218595 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.218110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37148)
0621 07:01:06.218346 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.218448 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260621 07:01:06.219843 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:06.219393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36487 (local address 127.0.0.1:37160)
0621 07:01:06.219619 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:06.219717 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260621 07:01:06.301337 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17748
I20260621 07:01:06.320447 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17395
I20260621 07:01:06.321615 17458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:06.434144 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17395
I20260621 07:01:06.457021 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--webserver_interface=127.2.245.254
--webserver_port=41669
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:06.780308 17922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:06.780649 17922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:06.780819 17922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:06.787743 17922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:01:06.787914 17922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:06.788026 17922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:06.788151 17922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:01:06.788230 17922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:01:06.797705 17922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=41669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:06.799069 17922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:06.800431 17922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:06.812006 17927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:06.812202 17928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:06.812232 17930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:06.813194 17922 server_base.cc:1034] running on GCE node
I20260621 07:01:06.814002 17922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:06.815426 17922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:06.816669 17922 hybrid_clock.cc:648] HybridClock initialized: now 1782025266816630 us; error 46 us; skew 500 ppm
I20260621 07:01:06.817044 17922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:06 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025266, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:06.820138 17922 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:01:06.821566 17922 webserver.cc:460] Webserver started at http://127.2.245.254:41669/ using document root <none> and password file <none>
I20260621 07:01:06.822271 17922 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:06.822419 17922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:06.828056 17922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:01:06.830871 17937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:06.832263 17922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:06.832439 17922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "631dc01aaf9d4e9b819863428136d264"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "5c1113167e9dc96c924e9825e0e69821"
server_key_iv: "71eec2470d8c64c691df7f5c87057e90"
server_key_version: "encryptionkey@0"
I20260621 07:01:06.833446 17922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:06.851131 17922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:06.855163 17922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:06.855419 17922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:06.873708 17922 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43509
I20260621 07:01:06.873728 17988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43509 every 8 connection(s)
I20260621 07:01:06.875191 17922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:01:06.878650 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17922
I20260621 07:01:06.879691 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:36487
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=39059
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:01:06.883468 17989 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:01:06.888382 17989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap starting.
I20260621 07:01:06.908560 17989 log.cc:824] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:06.924234 17989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:01:06.924748 17989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264: Bootstrap complete.
I20260621 07:01:06.933571 17989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:06.934497 17989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Initialized, Role: FOLLOWER
I20260621 07:01:06.935159 17989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:06.935395 17989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:01:06.935534 17989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:01:06.935720 17989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 2 FOLLOWER]: Advancing to term 3
I20260621 07:01:06.939833 17989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:06.940332 17989 leader_election.cc:302] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631dc01aaf9d4e9b819863428136d264; no voters:
I20260621 07:01:06.941004 17989 leader_election.cc:288] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260621 07:01:06.941136 17993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 3 FOLLOWER]: Leader election won for term 3
I20260621 07:01:06.942209 17993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [term 3 LEADER]: Becoming Leader. State: Replica: 631dc01aaf9d4e9b819863428136d264, State: Running, Role: LEADER
I20260621 07:01:06.942800 17993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } }
I20260621 07:01:06.943166 17989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:01:06.945533 17995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631dc01aaf9d4e9b819863428136d264. Latest consensus state: current_term: 3 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:06.945549 17994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "631dc01aaf9d4e9b819863428136d264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631dc01aaf9d4e9b819863428136d264" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43509 } } }
I20260621 07:01:06.945902 17995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:06.945911 17994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631dc01aaf9d4e9b819863428136d264 [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:06.958874 17998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:01:06.967818 17998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:01:06.969395 17998 catalog_manager.cc:1259] Loaded cluster ID: 620def677f10468e917425330d473a14
I20260621 07:01:06.969543 17998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260621 07:01:06.971442 17998 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0d6b57d9ad google::LogMessage::Fail() at ??:0
@ 0x7f0d6b58088a google::LogMessage::SendToLog() at ??:0
@ 0x7f0d6b57d4de google::LogMessage::Flush() at ??:0
@ 0x7f0d6b57ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0d80fb0512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0d80fad976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0d81035610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0d81035439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0d6e1c9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0d6e1cc91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
I20260621 07:01:07.014930 18012 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f0d6e1cc799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0d6e1a7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0d714816db start_thread at ??:0
@ 0x7f0d6a41761f clone at ??:0
W20260621 07:01:07.271741 17991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:07.272022 17991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:07.272173 17991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:07.279484 17991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:07.279611 17991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:07.279807 17991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:01:07.289525 17991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:36487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=39059
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:07.290925 17991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:07.292263 17991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:07.308115 18030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:07.308156 18033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:07.308182 18031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:07.309341 17991 server_base.cc:1034] running on GCE node
I20260621 07:01:07.309995 17991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:07.311028 17991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:07.312266 17991 hybrid_clock.cc:648] HybridClock initialized: now 1782025267312202 us; error 67 us; skew 500 ppm
I20260621 07:01:07.312615 17991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:07 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025267, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:07.315861 17991 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:01:07.317286 17991 webserver.cc:460] Webserver started at http://127.2.245.193:39059/ using document root <none> and password file <none>
I20260621 07:01:07.317941 17991 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:07.318041 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:07.323385 17991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:07.326309 18040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:07.327860 17991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:01:07.327999 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "9dca01673dd140289249555cd29d9f22"
format_stamp: "Formatted at 2026-06-21 07:01:01 on dist-test-slave-9zdv"
server_key: "2c3087147a9239f6cb84b4859b3802cc"
server_key_iv: "bc73fc54a460d437436e56f87d6af35d"
server_key_version: "encryptionkey@0"
I20260621 07:01:07.328925 17991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:07.353344 17991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:07.357877 17991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:07.358107 17991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:07.359185 17991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:07.360960 18047 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260621 07:01:07.366560 17991 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 07:01:07.366700 17991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260621 07:01:07.366843 17991 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 07:01:07.369902 17991 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 07:01:07.370028 17991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260621 07:01:07.370389 18047 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Bootstrap starting.
I20260621 07:01:07.378767 18047 log.cc:824] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:07.393738 18047 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:01:07.394341 18047 tablet_bootstrap.cc:492] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Bootstrap complete.
I20260621 07:01:07.394791 18047 ts_tablet_manager.cc:1397] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.008s
I20260621 07:01:07.405797 18047 raft_consensus.cc:348] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:07.407441 18047 raft_consensus.cc:729] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dca01673dd140289249555cd29d9f22, State: Initialized, Role: FOLLOWER
I20260621 07:01:07.408223 18047 consensus_queue.cc:260] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:07.409879 18047 ts_tablet_manager.cc:1428] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22: Time spent starting tablet: real 0.015s user 0.014s sys 0.004s
I20260621 07:01:07.414281 17991 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36487
I20260621 07:01:07.414366 18155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36487 every 8 connection(s)
I20260621 07:01:07.415603 17991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:01:07.416074 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 17991
I20260621 07:01:07.416934 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:38153
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=33477
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:01:07.420544 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.418452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:44861)
0621 07:01:07.419327 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.419623 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":380,"threads_started":1}
W20260621 07:01:07.421237 18156 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:43509 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:43509: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
I20260621 07:01:07.422092 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.421732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:51701)
0621 07:01:07.421908 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.421988 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:01:07.423111 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.422690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:48341)
0621 07:01:07.422850 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.422926 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 07:01:07.423383 18156 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:01:07.747901 18159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:07.748229 18159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:07.748410 18159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:07.755425 18159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:07.755580 18159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:07.755777 18159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:01:07.765086 18159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:38153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=33477
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:07.766322 18159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:07.767611 18159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:07.780465 18165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:07.780509 18164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:07.780536 18167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:07.781687 18159 server_base.cc:1034] running on GCE node
I20260621 07:01:07.782182 18159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:07.783110 18159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:07.784307 18159 hybrid_clock.cc:648] HybridClock initialized: now 1782025267784251 us; error 46 us; skew 500 ppm
I20260621 07:01:07.784564 18159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:07 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025267, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:07.787523 18159 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:01:07.788956 18159 webserver.cc:460] Webserver started at http://127.2.245.194:33477/ using document root <none> and password file <none>
I20260621 07:01:07.789619 18159 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:07.789795 18159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:07.794818 18159 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:07.799388 18174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:07.800678 18159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260621 07:01:07.800868 18159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "97361611225943309ca2ae08922d3f81"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "511269161b76b6416371522bccb0ae8d"
server_key_iv: "6aeff80d1dc5c22b0bb25a0c5b25d543"
server_key_version: "encryptionkey@0"
I20260621 07:01:07.801831 18159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:07.814570 18159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:07.818204 18159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:07.818532 18159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:07.819639 18159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:07.820943 18159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:07.821064 18159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:07.821189 18159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:07.821272 18159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:07.854048 18159 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38153
I20260621 07:01:07.854118 18286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38153 every 8 connection(s)
I20260621 07:01:07.855587 18159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:01:07.860232 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18159
I20260621 07:01:07.860319 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.858408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:45561)
0621 07:01:07.859182 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.859426 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
W20260621 07:01:07.861028 18287 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:43509 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:43509: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
I20260621 07:01:07.861017 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:37275
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:43509
--webserver_port=34663
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:01:07.861866 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.861563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:47267)
0621 07:01:07.861700 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.861763 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 07:01:07.862720 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:07.862396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:38243)
0621 07:01:07.862539 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:07.862617 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 07:01:07.862985 18287 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:01:08.182446 18290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:08.182719 18290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:08.182876 18290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:08.189989 18290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:08.190130 18290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:08.190311 18290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:01:08.199823 18290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:37275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=34663
--tserver_master_addrs=127.2.245.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:08.201131 18290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:08.202462 18290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:08.216687 18295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:08.216605 18298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:08.216638 18296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:08.217552 18290 server_base.cc:1034] running on GCE node
I20260621 07:01:08.218161 18290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:08.219192 18290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:08.220431 18290 hybrid_clock.cc:648] HybridClock initialized: now 1782025268220371 us; error 52 us; skew 500 ppm
I20260621 07:01:08.220808 18290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:08 dist-test-slave-9zdv krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025268, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:08.224041 18290 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:01:08.225557 18290 webserver.cc:460] Webserver started at http://127.2.245.195:34663/ using document root <none> and password file <none>
I20260621 07:01:08.226269 18290 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:08.226438 18290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:08.231720 18290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:01:08.234586 18305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:08.236003 18290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:08.236145 18290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "8dd8a512904b4c7a86c4dd67cc346b54"
format_stamp: "Formatted at 2026-06-21 07:01:02 on dist-test-slave-9zdv"
server_key: "6bcc1e1e7615690db26009aea8b15a8c"
server_key_iv: "551a7f0573398114cfabc25efc769e02"
server_key_version: "encryptionkey@0"
I20260621 07:01:08.237085 18290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:08.257686 18290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:08.261618 18290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:08.261839 18290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:08.262710 18290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:08.264020 18290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:08.264106 18290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:08.264178 18290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:08.264240 18290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:08.296412 18290 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37275
I20260621 07:01:08.296499 18417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37275 every 8 connection(s)
I20260621 07:01:08.297869 18290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:01:08.302474 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18290
I20260621 07:01:08.303110 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.300859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:34411)
0621 07:01:08.301660 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.301883 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":373,"threads_started":1}
W20260621 07:01:08.303794 18418 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:43509 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:43509: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
I20260621 07:01:08.304663 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.304353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:37285)
0621 07:01:08.304501 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.304561 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260621 07:01:08.305581 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.305262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:43259)
0621 07:01:08.305408 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.305479 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260621 07:01:08.305938 18418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 07:01:08.424587 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.423887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:35345)
0621 07:01:08.424312 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.424397 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":187,"threads_started":1}
I20260621 07:01:08.662514 18423 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:08.662880 18423 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:08.664548 18423 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
Jun 21 07:01:08 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025267, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
Jun 21 07:01:08 dist-test-slave-9zdv krb5kdc[16845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025267, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:01:08.691363 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.665156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:36153 (local address 127.2.245.194:38153)
0621 07:01:08.667502 (+ 2346us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:08.667522 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:08.667568 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:01:08.669908 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:08.669920 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:08.670434 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:08.670826 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:08.673502 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.675475 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.678277 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.679142 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.679293 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:08.683361 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:08.683371 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:08.683386 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:08.683432 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:08.687038 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688120 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:08.688128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:08.688136 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:08.688364 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688940 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:08.688945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:08.688948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:08.689627 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:08.689828 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:08.690557 (+ 729us) server_negotiation.cc:299] Negotiation successful
0621 07:01:08.690904 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2156,"thread_start_us":2334,"threads_started":1}
I20260621 07:01:08.691375 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.665228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:59871 (local address 127.2.245.195:37275)
0621 07:01:08.667791 (+ 2563us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:08.667802 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:08.667852 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:01:08.669934 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:08.669943 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:08.670435 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:08.670830 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:08.673502 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.675475 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.678294 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.679141 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.679341 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:08.683044 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:08.683054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:08.683069 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:08.683105 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:08.686917 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688119 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:08.688128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:08.688136 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:08.688364 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688904 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:08.688911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:08.688917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:08.689584 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:08.689737 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:08.690553 (+ 816us) server_negotiation.cc:299] Negotiation successful
0621 07:01:08.690907 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2388,"thread_start_us":2112,"threads_started":1}
I20260621 07:01:08.691388 18424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.665005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38153 (local address 127.2.245.193:36153)
0621 07:01:08.666832 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.666850 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:08.666976 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:08.671071 (+ 4095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:08.671083 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:08.672120 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:08.673274 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:08.673304 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.675688 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.675693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:08.678089 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:08.678110 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.679551 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.679556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:08.679685 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:08.680423 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:08.680440 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:08.683080 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:08.687167 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:08.687171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:08.687174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:08.687981 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:08.688486 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:08.688495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:08.688797 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:08.689798 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:08.689803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:08.690041 (+ 238us) client_negotiation.cc:769] Sending connection context
0621 07:01:08.690314 (+ 273us) client_negotiation.cc:240] Negotiation successful
0621 07:01:08.690877 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1704,"mutex_wait_us":295,"spinlock_wait_cycles":32512,"thread_start_us":1506,"threads_started":1}
I20260621 07:01:08.691501 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.664953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37275 (local address 127.2.245.193:59871)
0621 07:01:08.665297 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.665359 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:08.666008 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:08.671071 (+ 5063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:08.671083 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:08.672120 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:08.673274 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:08.673304 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.675685 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.675691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:08.678089 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:08.678110 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:08.679334 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:08.679341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:08.679598 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:08.680260 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:08.680282 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:08.682666 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:08.687108 (+ 4442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:08.687135 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:08.687152 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:08.687981 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:08.688486 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:08.688491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:08.688495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:08.688797 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:08.689737 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:08.689745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:08.690045 (+ 300us) client_negotiation.cc:769] Sending connection context
0621 07:01:08.690329 (+ 284us) client_negotiation.cc:240] Negotiation successful
0621 07:01:08.691076 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":559,"spinlock_wait_cycles":31360}
W20260621 07:01:08.694569 18042 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
W20260621 07:01:08.694808 18043 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:08.694991 18043 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dca01673dd140289249555cd29d9f22; no voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81
I20260621 07:01:08.695396 18423 raft_consensus.cc:2726] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:01:08.864346 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:08.863432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:54181)
0621 07:01:08.863956 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:08.864096 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":224,"threads_started":1}
I20260621 07:01:09.307389 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:09.306539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:48815)
0621 07:01:09.307059 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:09.307189 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":249,"threads_started":1}
I20260621 07:01:09.426216 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:09.425530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:36505)
0621 07:01:09.425973 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:09.426056 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":207,"threads_started":1}
I20260621 07:01:09.865926 18430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:09.865268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:37777)
0621 07:01:09.865688 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:09.865767 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":230,"threads_started":1}
I20260621 07:01:10.265432 18431 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:10.265679 18431 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:10.266311 18431 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
W20260621 07:01:10.267462 18042 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
W20260621 07:01:10.267652 18043 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:10.267793 18043 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dca01673dd140289249555cd29d9f22; no voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81
I20260621 07:01:10.268075 18431 raft_consensus.cc:2726] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:01:10.308993 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:10.308288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:54427)
0621 07:01:10.308735 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:10.308816 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":235,"threads_started":1}
I20260621 07:01:10.427836 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:10.427126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:59097)
0621 07:01:10.427581 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:10.427659 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":231,"threads_started":1}
I20260621 07:01:10.867617 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:10.866796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:42059)
0621 07:01:10.867324 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:10.867425 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":292,"threads_started":1}
I20260621 07:01:11.310672 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:11.309923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:50733)
0621 07:01:11.310431 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:11.310517 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":291,"threads_started":1}
I20260621 07:01:11.429409 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:11.428731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:56933)
0621 07:01:11.429170 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:11.429248 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":252,"threads_started":1}
I20260621 07:01:11.869180 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:11.868503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:33039)
0621 07:01:11.868916 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:11.868995 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":202,"threads_started":1}
I20260621 07:01:12.312497 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:12.311608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:47041)
0621 07:01:12.312050 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:12.312160 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":597,"threads_started":1}
I20260621 07:01:12.431286 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:12.430434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:34813)
0621 07:01:12.431011 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:12.431104 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":222,"threads_started":1}
I20260621 07:01:12.515463 18440 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:12.515728 18440 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:12.516467 18440 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
W20260621 07:01:12.517369 18043 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
W20260621 07:01:12.517591 18042 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:12.517746 18042 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dca01673dd140289249555cd29d9f22; no voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81
I20260621 07:01:12.518025 18440 raft_consensus.cc:2726] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:01:12.870792 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:12.870127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:57041)
0621 07:01:12.870550 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:12.870630 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":222,"threads_started":1}
I20260621 07:01:13.314054 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:13.313370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:51181)
0621 07:01:13.313813 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:13.313900 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":248,"threads_started":1}
I20260621 07:01:13.432857 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:13.432190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:35765)
0621 07:01:13.432613 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:13.432701 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":226,"threads_started":1}
I20260621 07:01:13.872380 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:13.871696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:59271)
0621 07:01:13.872135 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:13.872213 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":229,"threads_started":1}
I20260621 07:01:14.315810 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:14.314926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:41005)
0621 07:01:14.315500 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:14.315635 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":282,"threads_started":1}
I20260621 07:01:14.434482 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:14.433776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:57515)
0621 07:01:14.434194 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:14.434284 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":223,"threads_started":1}
I20260621 07:01:14.874132 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:14.873294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:52389)
0621 07:01:14.873877 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:14.873956 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":253,"threads_started":1}
I20260621 07:01:15.078437 18448 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:15.078613 18448 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:15.079312 18448 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
W20260621 07:01:15.080318 18043 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
W20260621 07:01:15.080539 18042 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:15.080650 18042 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dca01673dd140289249555cd29d9f22; no voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81
I20260621 07:01:15.080928 18448 raft_consensus.cc:2726] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:01:15.317544 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:15.316731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:49311)
0621 07:01:15.317246 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:15.317329 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":257,"threads_started":1}
I20260621 07:01:15.436242 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:15.435465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:52853)
0621 07:01:15.435967 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:15.436062 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":241,"threads_started":1}
I20260621 07:01:15.875988 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:15.875144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:56125)
0621 07:01:15.875697 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:15.875793 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":311,"threads_started":1}
I20260621 07:01:16.319159 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:16.318416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:51581)
0621 07:01:16.318812 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:16.318890 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":207,"threads_started":1}
I20260621 07:01:16.437853 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:16.437101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:33195)
0621 07:01:16.437581 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:16.437661 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260621 07:01:16.877653 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:16.876898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:42273)
0621 07:01:16.877397 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:16.877476 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
I20260621 07:01:17.327626 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:17.320205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:44961)
0621 07:01:17.327365 (+ 7160us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:17.327456 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7049,"thread_start_us":6917,"threads_started":1}
I20260621 07:01:17.439582 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:17.438803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:45931)
0621 07:01:17.439322 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:17.439401 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":302,"threads_started":1}
I20260621 07:01:17.879642 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:17.878651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:47777)
0621 07:01:17.879282 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:17.879407 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":365,"threads_started":1}
I20260621 07:01:18.329301 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:18.328595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:40949)
0621 07:01:18.329049 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:18.329130 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":245,"threads_started":1}
I20260621 07:01:18.441399 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:18.440486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:40329)
0621 07:01:18.441056 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:18.441191 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":309,"threads_started":1}
I20260621 07:01:18.881529 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:18.880680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:46159)
0621 07:01:18.881195 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:18.881290 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":233,"threads_started":1}
I20260621 07:01:19.331131 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:19.330292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:40585)
0621 07:01:19.330802 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:19.330888 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":274,"threads_started":1}
I20260621 07:01:19.443104 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:19.442360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:59415)
0621 07:01:19.442805 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:19.442889 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":231,"threads_started":1}
I20260621 07:01:19.883384 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:19.882603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:48339)
0621 07:01:19.883131 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:19.883214 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":239,"threads_started":1}
I20260621 07:01:20.332908 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:20.332181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:48577)
0621 07:01:20.332634 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:20.332735 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":226,"threads_started":1}
I20260621 07:01:20.444630 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:20.443928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:60595)
0621 07:01:20.444363 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:20.444458 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260621 07:01:20.885028 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:20.884309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:39341)
0621 07:01:20.884784 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:20.884874 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
I20260621 07:01:21.334628 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:21.333867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:39125)
0621 07:01:21.334348 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:21.334463 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":258,"threads_started":1}
I20260621 07:01:21.446224 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:21.445514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:37677)
0621 07:01:21.445954 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:21.446035 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260621 07:01:21.755517 18469 raft_consensus.cc:482] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:21.755759 18469 raft_consensus.cc:504] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97361611225943309ca2ae08922d3f81" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38153 } } peers { permanent_uuid: "8dd8a512904b4c7a86c4dd67cc346b54" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37275 } } peers { permanent_uuid: "9dca01673dd140289249555cd29d9f22" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36487 } }
I20260621 07:01:21.756358 18469 leader_election.cc:288] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153), 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275)
W20260621 07:01:21.757444 18043 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd8a512904b4c7a86c4dd67cc346b54 (127.2.245.195:37275): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
W20260621 07:01:21.757682 18042 leader_election.cc:341] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97361611225943309ca2ae08922d3f81 (127.2.245.194:38153): Not found: Tablet not found: d8f85e68553b440a8de37e60fe117bae
I20260621 07:01:21.757807 18042 leader_election.cc:302] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dca01673dd140289249555cd29d9f22; no voters: 8dd8a512904b4c7a86c4dd67cc346b54, 97361611225943309ca2ae08922d3f81
I20260621 07:01:21.758070 18469 raft_consensus.cc:2726] T d8f85e68553b440a8de37e60fe117bae P 9dca01673dd140289249555cd29d9f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:01:21.886641 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:21.885887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:54427)
0621 07:01:21.886360 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:21.886443 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260621 07:01:22.336426 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:22.335723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:43013)
0621 07:01:22.336138 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:22.336239 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":205,"threads_started":1}
I20260621 07:01:22.448099 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:22.447410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.193:48625)
0621 07:01:22.447841 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:22.447924 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":209,"threads_started":1}
I20260621 07:01:22.888345 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:22.887652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.194:58979)
0621 07:01:22.888079 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:22.888196 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":210,"threads_started":1}
I20260621 07:01:23.309252 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 17991
I20260621 07:01:23.324687 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:23.310316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.2.245.193:36487)
0621 07:01:23.311013 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:23.311023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:23.311056 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:01:23.311140 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:23.311149 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:23.311450 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:23.311637 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:23.313776 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.314783 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.316095 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.316669 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.316768 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:23.319269 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:23.319281 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:23.319295 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:23.319326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:23.322194 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.322796 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.322803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.322805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.322929 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.323308 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.323311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.323313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.323658 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:23.323884 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:23.324111 (+ 227us) server_negotiation.cc:299] Negotiation successful
0621 07:01:23.324302 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":347,"threads_started":1}
I20260621 07:01:23.325884 18151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:23.338209 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:23.337447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43509 (local address 127.2.245.195:35351)
0621 07:01:23.337962 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:23.338061 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:43509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":286,"threads_started":1}
I20260621 07:01:23.415975 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 17991
I20260621 07:01:23.433854 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18159
I20260621 07:01:23.447273 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:23.434666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.2.245.194:38153)
0621 07:01:23.435165 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:23.435169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:23.435186 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:01:23.435241 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:23.435245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:23.435303 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:23.435397 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:23.436683 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.437415 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.438646 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.439201 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.439299 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:23.441951 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:23.441955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:23.441959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:23.441989 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:23.444802 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.445443 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.445447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.445450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.445581 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.445937 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.445940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.445942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.446209 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:23.446405 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:23.446622 (+ 217us) server_negotiation.cc:299] Negotiation successful
0621 07:01:23.446801 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
I20260621 07:01:23.447956 18282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:23.533277 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18159
I20260621 07:01:23.550714 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18290
I20260621 07:01:23.565117 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:23.551776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.2.245.195:37275)
0621 07:01:23.552371 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:23.552376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:23.552394 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:01:23.552449 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:23.552452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:23.552521 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:23.552618 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:23.554046 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.555084 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.556332 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:23.556837 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:23.556945 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:23.559725 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:23.559730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:23.559734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:23.559768 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:23.562604 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.563320 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.563324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.563326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.563471 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:23.563807 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:23.563810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:23.563812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:23.564066 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:23.564279 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:23.564502 (+ 223us) server_negotiation.cc:299] Negotiation successful
0621 07:01:23.564687 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":337,"threads_started":1}
I20260621 07:01:23.566061 18413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:23.648422 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18290
2026-06-21T07:01:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24222 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:01:23 dist-test-slave-9zdv krb5kdc[18483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:01:23 dist-test-slave-9zdv krb5kdc[18483](info): set up 2 sockets
Jun 21 07:01:23 dist-test-slave-9zdv krb5kdc[18483](info): commencing operation
krb5kdc: starting...
W20260621 07:01:24.739665 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:01:24 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:01:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:01:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:01:24.968720 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:32929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:25.294168 18496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:25.294458 18496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:25.294538 18496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:25.301623 18496 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 07:01:25.301736 18496 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 07:01:25.301774 18496 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 07:01:25.301815 18496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:01:25.301862 18496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:25.301921 18496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:25.301965 18496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:01:25.302011 18496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:01:25.311918 18496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:32929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:25.313165 18496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:25.314419 18496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:25.326668 18504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:25.326737 18502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:25.326750 18501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:25.327056 18496 server_base.cc:1034] running on GCE node
I20260621 07:01:25.328130 18496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:25.329592 18496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:25.330838 18496 hybrid_clock.cc:648] HybridClock initialized: now 1782025285330766 us; error 54 us; skew 500 ppm
I20260621 07:01:25.331192 18496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:25 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025285, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:25.338297 18496 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:01:25.339905 18496 webserver.cc:460] Webserver started at http://127.2.245.254:43049/ using document root <none> and password file <none>
I20260621 07:01:25.340595 18496 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:25.340684 18496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:25.340955 18496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:25.344449 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "40fcd81d82a34620a4281dac2cfb37fe"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "e0651de0fdf27addd2ecda78b1d90ce6"
server_key_iv: "6e21a72db7e8f7222e2a54229178255e"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.345232 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "40fcd81d82a34620a4281dac2cfb37fe"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "e0651de0fdf27addd2ecda78b1d90ce6"
server_key_iv: "6e21a72db7e8f7222e2a54229178255e"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.350505 18496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260621 07:01:25.354086 18511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:25.355621 18496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 07:01:25.355800 18496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "40fcd81d82a34620a4281dac2cfb37fe"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "e0651de0fdf27addd2ecda78b1d90ce6"
server_key_iv: "6e21a72db7e8f7222e2a54229178255e"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.356027 18496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:25.393400 18496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:25.397658 18496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:25.397961 18496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:25.417418 18496 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:32929
I20260621 07:01:25.417440 18562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:32929 every 8 connection(s)
I20260621 07:01:25.418859 18496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:01:25.421459 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18496
I20260621 07:01:25.421711 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:01:25.422411 3031 external_mini_cluster.cc:1428] Setting key ca4f37cad7d850f7f8c6f0529bf326cc
I20260621 07:01:25.422984 18563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:25.437630 18563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap starting.
Jun 21 07:01:25 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:25.441654 18563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:25.443115 18563 log.cc:824] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:25.446722 18563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: No bootstrap required, opened a new log
I20260621 07:01:25.448293 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:25.424495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.2.245.254:32929)
0621 07:01:25.425343 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:25.425359 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:25.425409 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:01:25.428067 (+ 2658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:25.428096 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:25.428698 (+ 602us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:25.429187 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:25.430551 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:25.432577 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:25.433959 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:25.435142 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:25.435357 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:25.439502 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:25.439513 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:25.439534 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:25.439588 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:25.443562 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:25.444475 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:25.444481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:25.444487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:25.444736 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:25.445112 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:25.445116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:25.445119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:25.445829 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:25.446119 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:25.446490 (+ 371us) server_negotiation.cc:299] Negotiation successful
0621 07:01:25.446936 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":344,"threads_started":1}
I20260621 07:01:25.455333 18563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:25.455744 18563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:25.455821 18563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Initialized, Role: FOLLOWER
I20260621 07:01:25.456357 18563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:25.456557 18563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:01:25.456653 18563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:01:25.456776 18563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:25.459053 18563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:25.459626 18563 leader_election.cc:302] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40fcd81d82a34620a4281dac2cfb37fe; no voters:
I20260621 07:01:25.460248 18563 leader_election.cc:288] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:01:25.460397 18568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:25.461146 18568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 LEADER]: Becoming Leader. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Running, Role: LEADER
I20260621 07:01:25.461757 18568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:25.462461 18563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:01:25.464540 18570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40fcd81d82a34620a4281dac2cfb37fe. Latest consensus state: current_term: 1 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:01:25.464986 18570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:25.466257 18569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:01:25.466616 18569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:25.467774 18575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:01:25.474489 18575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:01:25.483673 18575 catalog_manager.cc:1347] Generated new cluster ID: 66a47441c47749478130cbd4f8d6eb2b
I20260621 07:01:25.483839 18575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:01:25.492341 18575 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:01:25.493638 18575 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:01:25.503856 18575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Generated new TSK 0
I20260621 07:01:25.504907 18575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:01:25.570864 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:25.907025 18589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:25.907366 18589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:25.907537 18589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:25.914783 18589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:25.914973 18589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:25.915162 18589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:01:25.925169 18589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:25.926471 18589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:25.927858 18589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:25.941039 18595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:25.941079 18594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:25.941581 18597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:25.941896 18589 server_base.cc:1034] running on GCE node
I20260621 07:01:25.942994 18589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:25.944097 18589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:25.945345 18589 hybrid_clock.cc:648] HybridClock initialized: now 1782025285945306 us; error 54 us; skew 500 ppm
I20260621 07:01:25.945708 18589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:25 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025285, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:25.948947 18589 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:01:25.950469 18589 webserver.cc:460] Webserver started at http://127.2.245.193:36335/ using document root <none> and password file <none>
I20260621 07:01:25.951243 18589 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:25.951390 18589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:25.951706 18589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:25.954558 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "a91d4168e714a199b48c3a14666224cd"
server_key_iv: "8fd5a8c3dde570fb9e360ca94e257c4a"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.955386 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "a91d4168e714a199b48c3a14666224cd"
server_key_iv: "8fd5a8c3dde570fb9e360ca94e257c4a"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.960608 18589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:01:25.964429 18604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:25.965895 18589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:01:25.966081 18589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "a91d4168e714a199b48c3a14666224cd"
server_key_iv: "8fd5a8c3dde570fb9e360ca94e257c4a"
server_key_version: "encryptionkey@0"
I20260621 07:01:25.966295 18589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:25.984285 18589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:25.988279 18589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:25.988561 18589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:25.989480 18589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:25.990816 18589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:25.990923 18589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:25.991096 18589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:25.991185 18589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:26.024914 18589 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:34101
I20260621 07:01:26.024964 18716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:34101 every 8 connection(s)
I20260621 07:01:26.026242 18589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:01:26.026757 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18589
I20260621 07:01:26.027036 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:01:26.027878 3031 external_mini_cluster.cc:1428] Setting key 83376b42cd3e8bb39ea6103e4c480ee7
Jun 21 07:01:26 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025285, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:26.047070 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:26.029029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41591 (local address 127.2.245.254:32929)
0621 07:01:26.029495 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:26.029499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:26.030756 (+ 1257us) server_negotiation.cc:407] Connection header received
0621 07:01:26.032573 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:26.032581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:26.032670 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:26.032775 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:26.034211 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.035379 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.036912 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.037385 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.037482 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:26.041502 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:26.041507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:26.041510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:26.041540 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:26.044066 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:26.044760 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:26.044768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:26.044772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:26.044895 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:26.045357 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:26.045364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:26.045367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:26.045640 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:26.045794 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:26.046453 (+ 659us) server_negotiation.cc:299] Negotiation successful
0621 07:01:26.046681 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":254,"threads_started":1}
I20260621 07:01:26.048097 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:26.029736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:41591)
0621 07:01:26.030567 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:26.030617 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:26.032306 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:26.032967 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:26.032975 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:26.033478 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:26.034040 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:26.034056 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.035556 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.035560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:26.036765 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:26.036776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.037624 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.037631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:26.037942 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:26.038742 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:26.038768 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:26.041260 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:26.044208 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:26.044217 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:26.044233 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:26.044635 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:26.045017 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:26.045022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:26.045026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:26.045233 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:26.045768 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:26.045775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:26.046106 (+ 331us) client_negotiation.cc:769] Sending connection context
0621 07:01:26.046378 (+ 272us) client_negotiation.cc:240] Negotiation successful
0621 07:01:26.046906 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":354,"threads_started":1}
I20260621 07:01:26.049715 18717 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:26.050050 18717 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:26.051014 18717 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
I20260621 07:01:26.053180 18528 ts_manager.cc:194] Registered new tserver with Master: 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
I20260621 07:01:26.055210 18528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:41591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 07:01:26.082463 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:26.407301 18723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:26.407596 18723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:26.407786 18723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:26.415198 18723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:26.415313 18723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:26.415513 18723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:01:26.425607 18723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:26.426995 18723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:26.428392 18723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:26.441812 18728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:26.441766 18729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:26.441841 18731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:26.442730 18723 server_base.cc:1034] running on GCE node
I20260621 07:01:26.443389 18723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:26.444428 18723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:26.445629 18723 hybrid_clock.cc:648] HybridClock initialized: now 1782025286445590 us; error 29 us; skew 500 ppm
I20260621 07:01:26.445955 18723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:26 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025286, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:26.449169 18723 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:01:26.450604 18723 webserver.cc:460] Webserver started at http://127.2.245.194:35905/ using document root <none> and password file <none>
I20260621 07:01:26.451355 18723 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:26.451501 18723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:26.451829 18723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:26.454725 18723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "88f0d6de23f14ef3900e9ef449865bb7"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "f2111eb431065ee67a3e3a08c76c3053"
server_key_iv: "846640c949bca2e785a2601d30dd37a1"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.455550 18723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "88f0d6de23f14ef3900e9ef449865bb7"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "f2111eb431065ee67a3e3a08c76c3053"
server_key_iv: "846640c949bca2e785a2601d30dd37a1"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.460651 18723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:01:26.464207 18738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:26.465508 18723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:01:26.465682 18723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "88f0d6de23f14ef3900e9ef449865bb7"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "f2111eb431065ee67a3e3a08c76c3053"
server_key_iv: "846640c949bca2e785a2601d30dd37a1"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.465898 18723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:26.480964 18723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:26.484844 18723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:26.485132 18723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:26.486080 18723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:26.487478 18723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:26.487582 18723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:26.487725 18723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:26.487818 18723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:26.522459 18723 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42263
I20260621 07:01:26.522472 18850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42263 every 8 connection(s)
I20260621 07:01:26.523904 18723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:01:26.525779 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18723
I20260621 07:01:26.525980 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:01:26.526458 3031 external_mini_cluster.cc:1428] Setting key d83b349e1b2c74cc50141022ed461a79
Jun 21 07:01:26 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025286, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:26.546288 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:26.526698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:47391 (local address 127.2.245.254:32929)
0621 07:01:26.526918 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:26.526924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:26.528465 (+ 1541us) server_negotiation.cc:407] Connection header received
0621 07:01:26.530384 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:26.530393 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:26.530485 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:26.530593 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:26.532100 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.533242 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.534781 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.535599 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.535750 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:26.540265 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:26.540270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:26.540273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:26.540307 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:26.543011 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:26.543929 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:26.543933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:26.543936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:26.544061 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:26.544556 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:26.544559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:26.544561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:26.544828 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:26.544936 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:26.545862 (+ 926us) server_negotiation.cc:299] Negotiation successful
0621 07:01:26.546045 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 07:01:26.547945 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:26.527360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:47391)
0621 07:01:26.528281 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:26.528336 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:26.530102 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:26.530793 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:26.530801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:26.531358 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:26.531927 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:26.531946 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.533386 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.533390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:26.534629 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:26.534641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:26.535849 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:26.535857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:26.536147 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:26.536934 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:26.536965 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:26.539991 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:26.543155 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:26.543167 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:26.543188 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:26.543807 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:26.544171 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:26.544179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:26.544185 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:26.544444 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:26.544956 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:26.544967 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:26.545448 (+ 481us) client_negotiation.cc:769] Sending connection context
0621 07:01:26.545761 (+ 313us) client_negotiation.cc:240] Negotiation successful
0621 07:01:26.546371 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":337,"threads_started":1}
I20260621 07:01:26.549644 18851 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:26.549968 18851 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:26.550712 18851 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
I20260621 07:01:26.552350 18528 ts_manager.cc:194] Registered new tserver with Master: 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
I20260621 07:01:26.553201 18528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:47391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:01:26.582293 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:26.906471 18856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:26.906731 18856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:26.906885 18856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:26.913976 18856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:26.914079 18856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:26.914285 18856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:01:26.924250 18856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:26.925554 18856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:26.926863 18856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:26.940788 18862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:26.940737 18861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:26.940824 18864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:26.941618 18856 server_base.cc:1034] running on GCE node
I20260621 07:01:26.942294 18856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:26.943385 18856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:26.944619 18856 hybrid_clock.cc:648] HybridClock initialized: now 1782025286944572 us; error 58 us; skew 500 ppm
I20260621 07:01:26.944947 18856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:26 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025286, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:26.948140 18856 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:01:26.949579 18856 webserver.cc:460] Webserver started at http://127.2.245.195:37921/ using document root <none> and password file <none>
I20260621 07:01:26.950253 18856 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:26.950352 18856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:26.950603 18856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:01:26.953590 18856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "fe0f237f28add408ae6bba25bbd4690b"
server_key_iv: "21cff75ffe522d2ce5e54cf92fe2ca5f"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.954339 18856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "fe0f237f28add408ae6bba25bbd4690b"
server_key_iv: "21cff75ffe522d2ce5e54cf92fe2ca5f"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.959834 18856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260621 07:01:26.963625 18871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:26.965032 18856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:01:26.965233 18856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "fe0f237f28add408ae6bba25bbd4690b"
server_key_iv: "21cff75ffe522d2ce5e54cf92fe2ca5f"
server_key_version: "encryptionkey@0"
I20260621 07:01:26.965456 18856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:27.000757 18856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:27.005020 18856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:27.005319 18856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:27.006304 18856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:27.007735 18856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:27.007836 18856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:27.007988 18856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:27.008061 18856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:27.041877 18856 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37297
I20260621 07:01:27.041926 18983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37297 every 8 connection(s)
I20260621 07:01:27.043257 18856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:01:27.045264 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 18856
I20260621 07:01:27.045461 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:01:27.046074 3031 external_mini_cluster.cc:1428] Setting key d42509550287fe228441900f91fe4321
Jun 21 07:01:27 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025286, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:27.058868 18717 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:27.065021 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.045937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:46113 (local address 127.2.245.254:32929)
0621 07:01:27.046112 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.046117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.047652 (+ 1535us) server_negotiation.cc:407] Connection header received
0621 07:01:27.049982 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.049987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.050080 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.050199 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:27.051749 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.052700 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.054554 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.055047 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.055186 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.058497 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:27.058502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:27.058505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:27.058536 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:27.062057 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.062813 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.062817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.062820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.063044 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.063500 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.063504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.063506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.063771 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:27.063891 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.064582 (+ 691us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.064765 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:01:27.065907 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.046605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:46113)
0621 07:01:27.047501 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.047550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.049727 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.050437 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.050444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.050991 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:27.051552 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.051568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.052881 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.052886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.054377 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.054391 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.055291 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.055299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.055537 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.056145 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:27.056192 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:27.058294 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:27.062231 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:27.062238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:27.062251 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:27.062685 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:27.063200 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:27.063205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:27.063209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:27.063395 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:27.063901 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:27.063906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:27.064240 (+ 334us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.064473 (+ 233us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.064840 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":377,"threads_started":1}
I20260621 07:01:27.067274 18984 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:27.067556 18984 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:27.068269 18984 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
I20260621 07:01:27.069692 18527 ts_manager.cc:194] Registered new tserver with Master: fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297)
I20260621 07:01:27.070914 18527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:46113
I20260621 07:01:27.074000 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:01:27.090306 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.078490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.2.245.254:32929)
0621 07:01:27.078656 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.078660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.079169 (+ 509us) server_negotiation.cc:407] Connection header received
0621 07:01:27.079307 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.079311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.079370 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.079449 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:27.080829 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.081517 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.082855 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.083357 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.083451 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.085768 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:27.085772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:27.085775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:27.085799 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:27.088146 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.088762 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.088766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.088768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.088880 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.089214 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.089217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.089219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.089500 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:27.089681 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.089925 (+ 244us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.090073 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:01:27.094475 18527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:01:27.101243 18527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:01:27.124877 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.112873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37297 (local address 127.0.0.1:45242)
0621 07:01:27.113668 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.113721 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.113916 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.117135 (+ 3219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.117139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.117160 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:27.117559 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.117573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.119571 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.119576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.121929 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.121941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.123826 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.123832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.124018 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124066 (+ 48us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.124185 (+ 119us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.124376 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":67,"thread_start_us":1368,"threads_started":1}
I20260621 07:01:27.125452 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.115950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.2.245.193:34101)
0621 07:01:27.117988 (+ 2038us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.117995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.118015 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:01:27.118082 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.118087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.118239 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.118421 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:27.119001 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.120472 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.122931 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.124064 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.124201 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124420 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.124968 (+ 548us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.125144 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1895,"thread_start_us":1756,"threads_started":1}
I20260621 07:01:27.125540 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.112952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42263 (local address 127.0.0.1:40320)
0621 07:01:27.115385 (+ 2433us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.115401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.115498 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.116461 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.116471 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.116504 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:27.116936 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.116947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.119515 (+ 2568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.119522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.122173 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.122188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.124689 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.124703 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.124900 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124932 (+ 32us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.125018 (+ 86us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.125144 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2292,"mutex_wait_us":87,"spinlock_wait_cycles":38784,"thread_start_us":2159,"threads_started":1}
I20260621 07:01:27.125993 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.115478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:58260)
0621 07:01:27.116439 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.116455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.116549 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.118410 (+ 1861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.118415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.118439 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:27.118767 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.118776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.120639 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.120648 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.122787 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.122798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.124285 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.124293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.124463 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124516 (+ 53us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.124615 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.124754 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":91,"thread_start_us":720,"threads_started":1}
I20260621 07:01:27.126124 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.113124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.2.245.195:37297)
0621 07:01:27.116052 (+ 2928us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.116062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.116097 (+ 35us) server_negotiation.cc:407] Connection header received
0621 07:01:27.116201 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.116222 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.116858 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.117130 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:27.117798 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.119406 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.122093 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.123582 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.123747 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124016 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.125622 (+ 1606us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.125788 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2772,"thread_start_us":2600,"threads_started":1}
I20260621 07:01:27.126233 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.113496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.2.245.194:42263)
0621 07:01:27.115544 (+ 2048us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.115557 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.115592 (+ 35us) server_negotiation.cc:407] Connection header received
0621 07:01:27.115709 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.115720 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.115979 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.116176 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:27.117719 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.119371 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.122334 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.124221 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.124353 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.124545 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.125764 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.125921 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1834,"thread_start_us":2423,"threads_started":1}
I20260621 07:01:27.129150 18652 tablet_service.cc:1467] Processing CreateTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 (DEFAULT_TABLE table=test-table [id=2c43e7f000eb499c9dda4a81f471650a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:27.129154 18786 tablet_service.cc:1467] Processing CreateTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 (DEFAULT_TABLE table=test-table [id=2c43e7f000eb499c9dda4a81f471650a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:27.130537 18652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0374654bc0c49cbbae2e0e87d783b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:27.130555 18786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0374654bc0c49cbbae2e0e87d783b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:27.132093 18919 tablet_service.cc:1467] Processing CreateTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 (DEFAULT_TABLE table=test-table [id=2c43e7f000eb499c9dda4a81f471650a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:27.133448 18919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0374654bc0c49cbbae2e0e87d783b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:27.141762 19002 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Bootstrap starting.
I20260621 07:01:27.144582 19003 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Bootstrap starting.
I20260621 07:01:27.146507 19002 tablet_bootstrap.cc:654] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:27.147094 19004 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Bootstrap starting.
I20260621 07:01:27.147856 19002 log.cc:824] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:27.148294 19003 tablet_bootstrap.cc:654] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:27.149843 19003 log.cc:824] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:27.149986 19004 tablet_bootstrap.cc:654] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:27.151463 19004 log.cc:824] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:27.152722 19002 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: No bootstrap required, opened a new log
I20260621 07:01:27.153014 19002 ts_tablet_manager.cc:1397] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260621 07:01:27.153220 19003 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: No bootstrap required, opened a new log
I20260621 07:01:27.153532 19003 ts_tablet_manager.cc:1397] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260621 07:01:27.154922 19004 tablet_bootstrap.cc:492] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: No bootstrap required, opened a new log
I20260621 07:01:27.155232 19004 ts_tablet_manager.cc:1397] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260621 07:01:27.160585 19003 raft_consensus.cc:348] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.161145 19003 raft_consensus.cc:374] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:27.161279 19003 raft_consensus.cc:729] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f0d6de23f14ef3900e9ef449865bb7, State: Initialized, Role: FOLLOWER
I20260621 07:01:27.161962 19003 consensus_queue.cc:260] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.164321 18851 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:27.164866 19003 ts_tablet_manager.cc:1428] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260621 07:01:27.163823 19002 raft_consensus.cc:348] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.165828 19002 raft_consensus.cc:374] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:27.165601 19004 raft_consensus.cc:348] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.165933 19002 raft_consensus.cc:729] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c4a517cd4d48f2a8ef677cb58b858a, State: Initialized, Role: FOLLOWER
I20260621 07:01:27.166035 19004 raft_consensus.cc:374] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:27.166133 19004 raft_consensus.cc:729] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce315c7ca6b4ab481ad80473bc36ef8, State: Initialized, Role: FOLLOWER
I20260621 07:01:27.166589 19002 consensus_queue.cc:260] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.166759 19004 consensus_queue.cc:260] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.167899 18984 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:27.168776 19004 ts_tablet_manager.cc:1428] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260621 07:01:27.170351 19002 ts_tablet_manager.cc:1428] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260621 07:01:27.191155 19010 raft_consensus.cc:482] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:27.191485 19010 raft_consensus.cc:504] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.193074 19010 leader_election.cc:288] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263), 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101)
I20260621 07:01:27.199484 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.193337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42263 (local address 127.2.245.195:55373)
0621 07:01:27.193535 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.193554 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.193660 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.194167 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.194173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.194224 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:27.194426 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.194440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.195743 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.195750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.197586 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.197601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.198844 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.198847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.199077 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.199101 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.199177 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.199278 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":43}
I20260621 07:01:27.199640 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.193482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:55373 (local address 127.2.245.194:42263)
0621 07:01:27.193670 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.193677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.193697 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:01:27.193790 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.193795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.193869 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.193970 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:27.194614 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.195585 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.197751 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.198692 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.198844 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.198919 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.199240 (+ 321us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.199379 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:01:27.200515 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.193740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.2.245.195:53369)
0621 07:01:27.194365 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:27.194383 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:27.194498 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:27.194838 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:27.194841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:27.194859 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:27.195215 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.195223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.196314 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.196320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.198118 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:27.198129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.199773 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.199781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:27.199987 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.200023 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 07:01:27.200121 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:01:27.200253 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1,"thread_start_us":316,"threads_started":1}
I20260621 07:01:27.200598 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.193851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:53369 (local address 127.2.245.193:34101)
0621 07:01:27.194098 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.194104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.194466 (+ 362us) server_negotiation.cc:407] Connection header received
0621 07:01:27.194630 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.194634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.194726 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.194821 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:27.195354 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.196167 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.198280 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.199472 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.199620 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.199701 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.200199 (+ 498us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.200361 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:01:27.200714 18806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0374654bc0c49cbbae2e0e87d783b53" candidate_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f0d6de23f14ef3900e9ef449865bb7" is_pre_election: true
I20260621 07:01:27.201247 18806 raft_consensus.cc:2454] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce315c7ca6b4ab481ad80473bc36ef8 in term 0.
I20260621 07:01:27.201506 18672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0374654bc0c49cbbae2e0e87d783b53" candidate_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" is_pre_election: true
I20260621 07:01:27.201920 18873 leader_election.cc:302] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8; no voters:
I20260621 07:01:27.202073 18672 raft_consensus.cc:2454] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce315c7ca6b4ab481ad80473bc36ef8 in term 0.
I20260621 07:01:27.202302 19010 raft_consensus.cc:2781] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:01:27.202463 19010 raft_consensus.cc:482] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:01:27.202585 19010 raft_consensus.cc:3037] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:27.204751 19010 raft_consensus.cc:504] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.205329 19010 leader_election.cc:288] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [CANDIDATE]: Term 1 election: Requested vote from peers 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263), 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101)
I20260621 07:01:27.205792 18806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0374654bc0c49cbbae2e0e87d783b53" candidate_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f0d6de23f14ef3900e9ef449865bb7"
I20260621 07:01:27.205847 18672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0374654bc0c49cbbae2e0e87d783b53" candidate_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
I20260621 07:01:27.206017 18672 raft_consensus.cc:3037] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:27.206012 18806 raft_consensus.cc:3037] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:27.209254 18806 raft_consensus.cc:2454] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce315c7ca6b4ab481ad80473bc36ef8 in term 1.
I20260621 07:01:27.209255 18672 raft_consensus.cc:2454] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce315c7ca6b4ab481ad80473bc36ef8 in term 1.
I20260621 07:01:27.209651 18873 leader_election.cc:302] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a, fce315c7ca6b4ab481ad80473bc36ef8; no voters:
I20260621 07:01:27.209952 19010 raft_consensus.cc:2781] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:27.210347 19010 raft_consensus.cc:686] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 LEADER]: Becoming Leader. State: Replica: fce315c7ca6b4ab481ad80473bc36ef8, State: Running, Role: LEADER
I20260621 07:01:27.210876 19010 consensus_queue.cc:237] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } }
I20260621 07:01:27.215356 18526 catalog_manager.cc:5495] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } health_report { overall_health: UNKNOWN } } }
I20260621 07:01:27.247047 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.242214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.2.245.195:37297)
0621 07:01:27.242411 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.242416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.242432 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:01:27.242600 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.242604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.242676 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.242796 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:27.243121 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.243827 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.245061 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.245468 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.245566 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.245978 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:27.246466 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:01:27.246580 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.246639 (+ 59us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.246701 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260621 07:01:27.254663 18672 raft_consensus.cc:1261] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Refusing update from remote peer fce315c7ca6b4ab481ad80473bc36ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:27.254813 18806 raft_consensus.cc:1261] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Refusing update from remote peer fce315c7ca6b4ab481ad80473bc36ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:27.255698 19012 consensus_queue.cc:1035] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:27.256047 19010 consensus_queue.cc:1035] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:27.285907 18526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:27.286226 18526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:27.289623 18526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Sending DeleteTablet for 3 replicas of tablet d0374654bc0c49cbbae2e0e87d783b53
I20260621 07:01:27.290740 18919 tablet_service.cc:1514] Processing DeleteTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:27 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:45242
I20260621 07:01:27.290895 18786 tablet_service.cc:1514] Processing DeleteTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:27 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:40320
I20260621 07:01:27.291035 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18589
I20260621 07:01:27.291882 18652 tablet_service.cc:1514] Processing DeleteTablet for tablet d0374654bc0c49cbbae2e0e87d783b53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:27 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:58260
I20260621 07:01:27.303136 19025 tablet_replica.cc:330] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: stopping tablet replica
I20260621 07:01:27.303862 19025 raft_consensus.cc:2229] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:01:27.304945 19025 raft_consensus.cc:2258] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:27.307318 19025 ts_tablet_manager.cc:1905] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:27.314146 19028 tablet_replica.cc:330] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: stopping tablet replica
I20260621 07:01:27.314539 19028 raft_consensus.cc:2229] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:27.315084 19028 raft_consensus.cc:2258] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:27.315397 19025 ts_tablet_manager.cc:1918] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:27.315640 19025 log.cc:1196] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/d0374654bc0c49cbbae2e0e87d783b53
I20260621 07:01:27.316236 19025 ts_tablet_manager.cc:1939] T d0374654bc0c49cbbae2e0e87d783b53 P fce315c7ca6b4ab481ad80473bc36ef8: Deleting consensus metadata
I20260621 07:01:27.318182 18514 catalog_manager.cc:4841] TS fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): tablet d0374654bc0c49cbbae2e0e87d783b53 (table test-table [id=2c43e7f000eb499c9dda4a81f471650a]) successfully deleted
Jun 21 07:01:27 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:01:27.319172 19026 tablet_replica.cc:330] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: stopping tablet replica
I20260621 07:01:27.319695 19026 raft_consensus.cc:2229] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:27.320264 19026 raft_consensus.cc:2258] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:27.320741 19028 ts_tablet_manager.cc:1905] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:27.323061 19026 ts_tablet_manager.cc:1905] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:27.327958 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.292388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.2.245.193:34101)
0621 07:01:27.292600 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.292608 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.305851 (+ 13243us) server_negotiation.cc:407] Connection header received
0621 07:01:27.306165 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.306172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.306258 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.306337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:27.308568 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.309726 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.311914 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.312620 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.312730 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.320080 (+ 7350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:27.320094 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:27.320110 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:27.320177 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:27.324235 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.325145 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.325153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.325157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.325324 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.325823 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.325828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.325832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.326311 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:27.326626 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.327297 (+ 671us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.327562 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:01:27.328032 19028 ts_tablet_manager.cc:1918] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:27.328421 19028 log.cc:1196] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/d0374654bc0c49cbbae2e0e87d783b53
I20260621 07:01:27.329233 19028 ts_tablet_manager.cc:1939] T d0374654bc0c49cbbae2e0e87d783b53 P 84c4a517cd4d48f2a8ef677cb58b858a: Deleting consensus metadata
I20260621 07:01:27.329579 18712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:27.330032 19026 ts_tablet_manager.cc:1918] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:27.330292 19026 log.cc:1196] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/d0374654bc0c49cbbae2e0e87d783b53
I20260621 07:01:27.330904 19026 ts_tablet_manager.cc:1939] T d0374654bc0c49cbbae2e0e87d783b53 P 88f0d6de23f14ef3900e9ef449865bb7: Deleting consensus metadata
I20260621 07:01:27.333036 18513 catalog_manager.cc:4841] TS 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): tablet d0374654bc0c49cbbae2e0e87d783b53 (table test-table [id=2c43e7f000eb499c9dda4a81f471650a]) successfully deleted
I20260621 07:01:27.454919 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18589
I20260621 07:01:27.455044 18513 catalog_manager.cc:4841] TS 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101): tablet d0374654bc0c49cbbae2e0e87d783b53 (table test-table [id=2c43e7f000eb499c9dda4a81f471650a]) successfully deleted
I20260621 07:01:27.473901 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18723
Jun 21 07:01:27 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:01:27.490896 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.474755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.2.245.194:42263)
0621 07:01:27.475130 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.475134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.475149 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:01:27.475425 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.475431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.475523 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.475638 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:27.476965 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.478165 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.479580 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.480219 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.480353 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.484398 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:27.484414 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:27.484432 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:27.484478 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:27.488044 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.488752 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.488762 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.488766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.488926 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.489285 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.489291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.489296 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.489773 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:27.490062 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.490320 (+ 258us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.490530 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260621 07:01:27.491613 18846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:27.581836 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18723
I20260621 07:01:27.605332 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18856
Jun 21 07:01:27 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:01:27.622612 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:27.606081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.2.245.195:37297)
0621 07:01:27.606282 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:27.606287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:27.606310 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:01:27.606375 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:27.606380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:27.606487 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:27.606595 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:27.608012 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.609266 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.610594 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:27.611319 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:27.611455 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:27.615619 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:27.615633 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:27.615651 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:27.615694 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:27.619550 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.620298 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.620309 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.620312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.620509 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:27.620900 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:27.620906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:27.620908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:27.621400 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:27.621699 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:27.621935 (+ 236us) server_negotiation.cc:299] Negotiation successful
0621 07:01:27.622161 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 07:01:27.623430 18979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:27.746541 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18856
I20260621 07:01:27.765894 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 18496
I20260621 07:01:27.767076 18558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:27.875964 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 18496
I20260621 07:01:47.897154 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--webserver_interface=127.2.245.254
--webserver_port=43049
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:32929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:01:48.228331 19033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:48.228615 19033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:48.228785 19033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:48.236125 19033 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 07:01:48.236235 19033 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 07:01:48.236310 19033 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 07:01:48.236400 19033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:01:48.236481 19033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:48.236603 19033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:48.236686 19033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:01:48.236759 19033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:01:48.246666 19033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:32929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=43049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:48.248013 19033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:48.249334 19033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:48.272022 19040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:48.272135 19042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:48.272095 19039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:48.272858 19033 server_base.cc:1034] running on GCE node
I20260621 07:01:48.273761 19033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:48.275422 19033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:48.276669 19033 hybrid_clock.cc:648] HybridClock initialized: now 1782025308276579 us; error 64 us; skew 500 ppm
I20260621 07:01:48.277043 19033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:48 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025308, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:48.284142 19033 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:01:48.285871 19033 webserver.cc:460] Webserver started at http://127.2.245.254:43049/ using document root <none> and password file <none>
I20260621 07:01:48.286646 19033 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:48.286762 19033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:48.292825 19033 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260621 07:01:48.295807 19049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:48.297175 19033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:01:48.297367 19033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "40fcd81d82a34620a4281dac2cfb37fe"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "e0651de0fdf27addd2ecda78b1d90ce6"
server_key_iv: "6e21a72db7e8f7222e2a54229178255e"
server_key_version: "encryptionkey@0"
I20260621 07:01:48.298427 19033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:48.322475 19033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:48.327150 19033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:48.327454 19033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:48.346483 19033 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:32929
I20260621 07:01:48.346495 19100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:32929 every 8 connection(s)
I20260621 07:01:48.348088 19033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:01:48.350919 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19033
I20260621 07:01:48.351935 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:34101
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=36335
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:01:48.358071 19101 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:01:48.362807 19101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap starting.
I20260621 07:01:48.383486 19101 log.cc:824] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:48.396623 19101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:01:48.397209 19101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap complete.
I20260621 07:01:48.406890 19101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:48.407898 19101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Initialized, Role: FOLLOWER
I20260621 07:01:48.408548 19101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:48.408803 19101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:01:48.408946 19101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:01:48.409135 19101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:01:48.413256 19101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:48.413801 19101 leader_election.cc:302] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40fcd81d82a34620a4281dac2cfb37fe; no voters:
I20260621 07:01:48.414494 19101 leader_election.cc:288] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:01:48.414631 19105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:01:48.415875 19105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 LEADER]: Becoming Leader. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Running, Role: LEADER
I20260621 07:01:48.416520 19105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:01:48.416917 19101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:01:48.419379 19107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40fcd81d82a34620a4281dac2cfb37fe. Latest consensus state: current_term: 2 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:01:48.419778 19107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:48.419427 19106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:01:48.419922 19106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:01:48.425295 19111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:01:48.435056 19111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:01:48.436970 19111 catalog_manager.cc:1259] Loaded cluster ID: 66a47441c47749478130cbd4f8d6eb2b
I20260621 07:01:48.437139 19111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:01:48.442600 19111 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:01:48.444713 19111 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Loaded TSK: 0
I20260621 07:01:48.450197 19122 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:01:48.470122 19111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Generated new TSK 1
I20260621 07:01:48.470491 19111 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:01:48.754403 19103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:48.754715 19103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:48.754884 19103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:48.762267 19103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:48.762442 19103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:48.762629 19103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:01:48.773470 19103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:34101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=36335
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:48.774797 19103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:48.776476 19103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:48.790493 19128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:48.791188 19129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:48.791211 19131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:48.791791 19103 server_base.cc:1034] running on GCE node
I20260621 07:01:48.792490 19103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:48.793767 19103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:48.795034 19103 hybrid_clock.cc:648] HybridClock initialized: now 1782025308794960 us; error 54 us; skew 500 ppm
I20260621 07:01:48.795385 19103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:48 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025308, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:48.799041 19103 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:01:48.800896 19103 webserver.cc:460] Webserver started at http://127.2.245.193:36335/ using document root <none> and password file <none>
I20260621 07:01:48.801851 19103 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:48.802019 19103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:48.809484 19103 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:01:48.813203 19138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:48.815060 19103 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:01:48.815244 19103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "a91d4168e714a199b48c3a14666224cd"
server_key_iv: "8fd5a8c3dde570fb9e360ca94e257c4a"
server_key_version: "encryptionkey@0"
I20260621 07:01:48.816668 19103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:48.843850 19103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:48.849406 19103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:48.849699 19103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:48.851024 19103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:48.852707 19103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:48.852818 19103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:48.852931 19103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:48.853019 19103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:48.890514 19103 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:34101
I20260621 07:01:48.890559 19250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:34101 every 8 connection(s)
I20260621 07:01:48.892307 19103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:01:48.900342 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19103
I20260621 07:01:48.901117 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:42263
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=35905
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:01:48 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025308, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:48.917671 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:48.895647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:34979)
0621 07:01:48.896438 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:48.896482 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:48.898229 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:48.899819 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:48.899832 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:48.900504 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:48.901077 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:48.901094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:48.903134 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:48.903141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:48.904410 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:48.904423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:48.905536 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:48.905544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:48.905830 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:48.906541 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:48.906568 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:48.908883 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:48.913180 (+ 4297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:48.913188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:48.913199 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:48.913647 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:48.914233 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:48.914237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:48.914239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:48.914437 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:48.915468 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:48.915475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:48.915801 (+ 326us) client_negotiation.cc:769] Sending connection context
0621 07:01:48.916043 (+ 242us) client_negotiation.cc:240] Negotiation successful
0621 07:01:48.916468 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":322,"threads_started":1}
I20260621 07:01:48.918418 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:48.895310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:34979 (local address 127.2.245.254:32929)
0621 07:01:48.895937 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:48.895949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:48.896622 (+ 673us) server_negotiation.cc:407] Connection header received
0621 07:01:48.898803 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:48.898815 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:48.899264 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:48.899650 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:48.901318 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:48.902847 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:48.904572 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:48.905267 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:48.905478 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:48.909232 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:48.909241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:48.909258 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:48.909303 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:48.912925 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:48.913801 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:48.913813 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:48.913824 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:48.914036 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:48.914558 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:48.914566 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:48.914570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:48.915310 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:48.915503 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:48.916309 (+ 806us) server_negotiation.cc:299] Negotiation successful
0621 07:01:48.916807 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":279,"threads_started":1}
I20260621 07:01:48.921166 19251 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:48.921515 19251 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:48.922447 19251 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
I20260621 07:01:48.924925 19066 ts_manager.cc:194] Registered new tserver with Master: 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101)
I20260621 07:01:48.926555 19066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:34979
W20260621 07:01:49.236481 19255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:49.236780 19255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:49.236943 19255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:49.244433 19255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:49.244555 19255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:49.244755 19255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:01:49.254789 19255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:42263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=35905
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:49.256102 19255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:49.257429 19255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:49.270782 19260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:49.270808 19261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:49.271008 19263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:49.271654 19255 server_base.cc:1034] running on GCE node
I20260621 07:01:49.272336 19255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:49.273304 19255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:49.274569 19255 hybrid_clock.cc:648] HybridClock initialized: now 1782025309274518 us; error 61 us; skew 500 ppm
I20260621 07:01:49.274923 19255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:49 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025309, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:49.278309 19255 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:01:49.279870 19255 webserver.cc:460] Webserver started at http://127.2.245.194:35905/ using document root <none> and password file <none>
I20260621 07:01:49.280581 19255 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:49.280756 19255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:49.286230 19255 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:01:49.288985 19270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.290417 19255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:01:49.290604 19255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "88f0d6de23f14ef3900e9ef449865bb7"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "f2111eb431065ee67a3e3a08c76c3053"
server_key_iv: "846640c949bca2e785a2601d30dd37a1"
server_key_version: "encryptionkey@0"
I20260621 07:01:49.291752 19255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:49.308817 19255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:49.313184 19255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:49.313464 19255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:49.314446 19255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:49.315860 19255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:49.315976 19255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.316099 19255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:49.316192 19255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.353935 19255 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42263
I20260621 07:01:49.354025 19382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42263 every 8 connection(s)
I20260621 07:01:49.355465 19255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:01:49.363163 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19255
I20260621 07:01:49.363946 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:37297
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=37921
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:01:49 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025309, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:49.378916 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.358328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:48177 (local address 127.2.245.254:32929)
0621 07:01:49.358509 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.358515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.360300 (+ 1785us) server_negotiation.cc:407] Connection header received
0621 07:01:49.362335 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.362341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.362465 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.362585 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:49.364827 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.365549 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.368109 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.368535 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.368670 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.372920 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:49.372925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:49.372929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:49.372969 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:49.375644 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.376554 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.376558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.376561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.376698 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.377146 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.377156 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.377159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.377434 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:49.377601 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.378314 (+ 713us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.378557 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:01:49.379976 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.359299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:48177)
0621 07:01:49.360128 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:49.360173 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:49.362033 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:49.362920 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:49.362964 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:49.363740 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:49.364586 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.364613 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.365734 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.365742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.367940 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.367959 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.368782 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.368791 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.369133 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.369908 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:49.369944 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:49.372657 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:49.375797 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:49.375809 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:49.375827 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:49.376412 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:49.376822 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:49.376827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:49.376830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:49.377019 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:49.377573 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:49.377581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:49.377940 (+ 359us) client_negotiation.cc:769] Sending connection context
0621 07:01:49.378217 (+ 277us) client_negotiation.cc:240] Negotiation successful
0621 07:01:49.378768 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":425,"threads_started":1}
I20260621 07:01:49.381635 19383 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:49.381989 19383 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:49.382755 19383 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
I20260621 07:01:49.384447 19066 ts_manager.cc:194] Registered new tserver with Master: 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
I20260621 07:01:49.385798 19066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:48177
W20260621 07:01:49.701732 19386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:01:49.702013 19386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:01:49.702106 19386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:01:49.709585 19386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:01:49.709695 19386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:01:49.709816 19386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:01:49.719700 19386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:37297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=37921
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:01:49.721009 19386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:01:49.722296 19386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:01:49.735746 19391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:49.735786 19394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:01:49.735852 19392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:01:49.736739 19386 server_base.cc:1034] running on GCE node
I20260621 07:01:49.737413 19386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:01:49.738384 19386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:01:49.739602 19386 hybrid_clock.cc:648] HybridClock initialized: now 1782025309739552 us; error 44 us; skew 500 ppm
I20260621 07:01:49.739965 19386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:01:49 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025309, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:01:49.743208 19386 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:01:49.744556 19386 webserver.cc:460] Webserver started at http://127.2.245.195:37921/ using document root <none> and password file <none>
I20260621 07:01:49.745178 19386 fs_manager.cc:359] Metadata directory not provided
I20260621 07:01:49.745289 19386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:01:49.750416 19386 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:01:49.753122 19401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.754496 19386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:01:49.754669 19386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "fe0f237f28add408ae6bba25bbd4690b"
server_key_iv: "21cff75ffe522d2ce5e54cf92fe2ca5f"
server_key_version: "encryptionkey@0"
I20260621 07:01:49.755746 19386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:01:49.780712 19386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:01:49.785152 19386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:01:49.785431 19386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:01:49.786564 19386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:01:49.787997 19386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:01:49.788092 19386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.788251 19386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:01:49.788316 19386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:01:49.821764 19386 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37297
I20260621 07:01:49.821813 19513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37297 every 8 connection(s)
I20260621 07:01:49.823398 19386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:01:49.831276 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19386
Jun 21 07:01:49 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025309, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:01:49.845324 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.826023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:49127 (local address 127.2.245.254:32929)
0621 07:01:49.826199 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.826204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.827476 (+ 1272us) server_negotiation.cc:407] Connection header received
0621 07:01:49.829247 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.829255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.829346 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.829465 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:49.830880 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.831730 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.833842 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.834274 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.834403 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.838150 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:49.838156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:49.838161 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:49.838200 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:49.841534 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.842458 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.842462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.842465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.842592 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.843111 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.843114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.843116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.843532 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:49.843678 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.844714 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.844938 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":768}
I20260621 07:01:49.846484 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.826558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:49127)
0621 07:01:49.827304 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:49.827347 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:49.828974 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:49.829684 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:49.829694 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:49.830179 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:01:49.830717 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.830733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.831894 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.831900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.833708 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.833720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.834499 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.834507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.834725 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.835437 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:01:49.835464 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:01:49.837938 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:01:49.841675 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:49.841686 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:49.841704 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:49.842333 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:49.842702 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:01:49.842707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:01:49.842711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:01:49.842987 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:01:49.843764 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:01:49.843777 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:01:49.844314 (+ 537us) client_negotiation.cc:769] Sending connection context
0621 07:01:49.844565 (+ 251us) client_negotiation.cc:240] Negotiation successful
0621 07:01:49.844951 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":264,"threads_started":1}
I20260621 07:01:49.847975 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.832183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.2.245.254:32929)
0621 07:01:49.832672 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.832678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.833054 (+ 376us) server_negotiation.cc:407] Connection header received
0621 07:01:49.833178 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.833182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.833255 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.833353 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:49.834840 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.835888 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.837174 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.837834 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.837963 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.841441 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:49.841446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:49.841450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:49.841515 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:49.844918 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.845825 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.845829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.845831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.845936 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.846340 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.846344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.846346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.846675 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:49.846881 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.847328 (+ 447us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.847532 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"spinlock_wait_cycles":6016,"thread_start_us":231,"threads_started":1}
I20260621 07:01:49.848119 19514 heartbeater.cc:347] Connected to a master server at 127.2.245.254:32929
I20260621 07:01:49.848485 19514 heartbeater.cc:464] Registering TS with master...
I20260621 07:01:49.849187 19514 heartbeater.cc:511] Master 127.2.245.254:32929 requested a full tablet report, sending...
I20260621 07:01:49.850764 19066 ts_manager.cc:194] Registered new tserver with Master: fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297)
I20260621 07:01:49.851589 19066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:49127
I20260621 07:01:49.862757 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:01:49.869540 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.863711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.2.245.254:32929)
0621 07:01:49.863900 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.863905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.864392 (+ 487us) server_negotiation.cc:407] Connection header received
0621 07:01:49.864565 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.864568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.864696 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.864819 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:49.865282 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.866007 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.867650 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.868107 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.868212 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.868765 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:49.869015 (+ 250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0621 07:01:49.869348 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37298: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260621 07:01:49.869689 19517 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37298: token expired
W20260621 07:01:49.870263 18988 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.245.254:32929)
I20260621 07:01:49.883786 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.870835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.2.245.254:32929)
0621 07:01:49.871040 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.871046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.871065 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:01:49.871139 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.871144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.871217 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.871321 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:49.872691 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.873397 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.874853 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.875357 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.875501 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.878180 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:49.878188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:49.878198 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:49.878231 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:49.880905 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.881695 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.881701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.881710 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.881838 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:49.882223 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:49.882229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:49.882232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:49.882541 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:49.882810 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.883189 (+ 379us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.883431 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:01:49.887364 19066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:01:49.894815 19066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:01:49.928557 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.907130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36554)
0621 07:01:49.918293 (+ 11163us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:49.918336 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:49.918524 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:49.919517 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:49.919533 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:49.919560 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:49.919917 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.919927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.922190 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.922198 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.925208 (+ 3010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.925224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.927440 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.927448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.927706 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.927790 (+ 84us) client_negotiation.cc:769] Sending connection context
0621 07:01:49.927905 (+ 115us) client_negotiation.cc:240] Negotiation successful
0621 07:01:49.928119 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10990,"mutex_wait_us":76,"thread_start_us":10841,"threads_started":1}
I20260621 07:01:49.928982 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.907007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.2.245.195:37297)
0621 07:01:49.917707 (+ 10700us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.917718 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.918447 (+ 729us) server_negotiation.cc:407] Connection header received
0621 07:01:49.918771 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.918780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.919069 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.919408 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:49.920688 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.922895 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.926087 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.927730 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.927885 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.928139 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.928507 (+ 368us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.928679 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10439,"thread_start_us":10249,"threads_started":1}
I20260621 07:01:49.929781 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.906825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37297 (local address 127.0.0.1:58768)
0621 07:01:49.918295 (+ 11470us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:49.918331 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:49.918562 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:49.919923 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:49.919931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:49.919982 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:49.920453 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.920468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.923117 (+ 2649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.923123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.925881 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.925896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.927931 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.927936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.928112 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.928147 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 07:01:49.929168 (+ 1021us) client_negotiation.cc:240] Negotiation successful
0621 07:01:49.929316 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11170,"mutex_wait_us":1,"thread_start_us":10901,"threads_started":1}
I20260621 07:01:49.930410 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.907437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.2.245.193:34101)
0621 07:01:49.917220 (+ 9783us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.917233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.918745 (+ 1512us) server_negotiation.cc:407] Connection header received
0621 07:01:49.918858 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.918865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.919131 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.920107 (+ 976us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:49.920293 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.921974 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.925399 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.927140 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.928284 (+ 1144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.929590 (+ 1306us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.929678 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.929863 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9590,"thread_start_us":9422,"threads_started":1}
I20260621 07:01:49.930670 19251 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:49.934135 19449 tablet_service.cc:1467] Processing CreateTablet for tablet 511e19852b1e4ebe9224adb60d58178a (DEFAULT_TABLE table=test-table [id=33b5759ac5b44067a8555549b460c34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:49.934399 19186 tablet_service.cc:1467] Processing CreateTablet for tablet 511e19852b1e4ebe9224adb60d58178a (DEFAULT_TABLE table=test-table [id=33b5759ac5b44067a8555549b460c34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:49.935401 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.920824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42263 (local address 127.0.0.1:53718)
0621 07:01:49.926247 (+ 5423us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:49.926262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:49.926428 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:49.927196 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:49.927200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:49.927224 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:49.927616 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.927625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.929489 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.929495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.932486 (+ 2991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:49.932507 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.934518 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.934525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:49.934701 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.934738 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 07:01:49.934839 (+ 101us) client_negotiation.cc:240] Negotiation successful
0621 07:01:49.934995 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5304,"mutex_wait_us":5,"thread_start_us":5136,"threads_started":1}
I20260621 07:01:49.935786 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:49.918928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.2.245.194:42263)
0621 07:01:49.924740 (+ 5812us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:49.924749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:49.926373 (+ 1624us) server_negotiation.cc:407] Connection header received
0621 07:01:49.926619 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:49.926629 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:49.926992 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:49.927195 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:49.927794 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.929320 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.932700 (+ 3380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:49.934273 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:49.934450 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:49.934663 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:49.935276 (+ 613us) server_negotiation.cc:299] Negotiation successful
0621 07:01:49.935462 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5620,"thread_start_us":5524,"threads_started":1}
I20260621 07:01:49.935832 19449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511e19852b1e4ebe9224adb60d58178a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:49.936177 19186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511e19852b1e4ebe9224adb60d58178a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:49.939558 19318 tablet_service.cc:1467] Processing CreateTablet for tablet 511e19852b1e4ebe9224adb60d58178a (DEFAULT_TABLE table=test-table [id=33b5759ac5b44067a8555549b460c34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:01:49.940992 19318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511e19852b1e4ebe9224adb60d58178a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:01:49.949196 19530 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Bootstrap starting.
I20260621 07:01:49.953127 19530 tablet_bootstrap.cc:654] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:49.954237 19530 log.cc:824] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:49.957362 19530 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: No bootstrap required, opened a new log
I20260621 07:01:49.957460 19533 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Bootstrap starting.
I20260621 07:01:49.957702 19530 ts_tablet_manager.cc:1397] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260621 07:01:49.959165 19531 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Bootstrap starting.
I20260621 07:01:49.960752 19533 tablet_bootstrap.cc:654] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:49.961786 19533 log.cc:824] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:49.963056 19531 tablet_bootstrap.cc:654] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Neither blocks nor log segments found. Creating new log.
I20260621 07:01:49.964335 19531 log.cc:824] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Log is configured to *not* fsync() on all Append() calls
I20260621 07:01:49.964421 19533 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: No bootstrap required, opened a new log
I20260621 07:01:49.964694 19533 ts_tablet_manager.cc:1397] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260621 07:01:49.971493 19530 raft_consensus.cc:348] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.972082 19530 raft_consensus.cc:374] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:49.972206 19530 raft_consensus.cc:729] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c4a517cd4d48f2a8ef677cb58b858a, State: Initialized, Role: FOLLOWER
I20260621 07:01:49.972095 19533 raft_consensus.cc:348] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.972566 19533 raft_consensus.cc:374] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:49.972741 19533 raft_consensus.cc:729] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f0d6de23f14ef3900e9ef449865bb7, State: Initialized, Role: FOLLOWER
I20260621 07:01:49.973065 19530 consensus_queue.cc:260] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.973294 19533 consensus_queue.cc:260] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.974977 19383 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:49.975621 19533 ts_tablet_manager.cc:1428] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260621 07:01:49.976173 19531 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: No bootstrap required, opened a new log
I20260621 07:01:49.976382 19530 ts_tablet_manager.cc:1428] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent starting tablet: real 0.019s user 0.016s sys 0.002s
I20260621 07:01:49.976521 19531 ts_tablet_manager.cc:1397] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.011s
I20260621 07:01:49.985385 19531 raft_consensus.cc:348] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.985733 19531 raft_consensus.cc:374] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:01:49.985795 19531 raft_consensus.cc:729] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce315c7ca6b4ab481ad80473bc36ef8, State: Initialized, Role: FOLLOWER
I20260621 07:01:49.986249 19531 consensus_queue.cc:260] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:49.987406 19514 heartbeater.cc:503] Master 127.2.245.254:32929 was elected leader, sending a full tablet report...
I20260621 07:01:49.987844 19531 ts_tablet_manager.cc:1428] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260621 07:01:50.074968 19515 tablet.cc:2369] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:01:50.110299 19384 tablet.cc:2369] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:01:50.144830 19252 tablet.cc:2369] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:01:50.275914 19536 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:01:50.276283 19536 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:50.278172 19536 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
I20260621 07:01:50.285188 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.278513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:58207 (local address 127.2.245.195:37297)
0621 07:01:50.278753 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.278762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.279896 (+ 1134us) server_negotiation.cc:407] Connection header received
0621 07:01:50.280009 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.280016 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.280102 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.280226 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:50.280826 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.281533 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.283449 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.284218 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.284396 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.284470 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.284818 (+ 348us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.284952 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:01:50.285305 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.278456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37297 (local address 127.2.245.193:58207)
0621 07:01:50.279757 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.279775 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:50.279892 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:50.280235 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:50.280238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:50.280256 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:50.280634 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:50.280662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.281666 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.281670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:50.283280 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:50.283293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.284415 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.284421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:50.284632 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.284664 (+ 32us) client_negotiation.cc:769] Sending connection context
0621 07:01:50.284767 (+ 103us) client_negotiation.cc:240] Negotiation successful
0621 07:01:50.284896 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"mutex_wait_us":29,"thread_start_us":931,"threads_started":1}
I20260621 07:01:50.285370 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.279062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41461 (local address 127.2.245.194:42263)
0621 07:01:50.279249 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.279253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.279378 (+ 125us) server_negotiation.cc:407] Connection header received
0621 07:01:50.279651 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.279657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.279740 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.279849 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:01:50.280605 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.281454 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.283445 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.284284 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.284434 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.284504 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.284989 (+ 485us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.285123 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:01:50.286011 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.278449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42263 (local address 127.2.245.193:41461)
0621 07:01:50.279231 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.279259 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0621 07:01:50.279458 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:01:50.279932 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:01:50.279938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:01:50.279971 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:01:50.280449 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:50.280464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.281592 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.281601 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:50.283280 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:01:50.283293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.284630 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.284635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:01:50.284820 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.284849 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 07:01:50.285528 (+ 679us) client_negotiation.cc:240] Negotiation successful
0621 07:01:50.285661 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":63,"thread_start_us":361,"threads_started":1}
I20260621 07:01:50.286363 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511e19852b1e4ebe9224adb60d58178a" candidate_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" is_pre_election: true
I20260621 07:01:50.286878 19469 raft_consensus.cc:2454] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c4a517cd4d48f2a8ef677cb58b858a in term 0.
I20260621 07:01:50.287009 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511e19852b1e4ebe9224adb60d58178a" candidate_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f0d6de23f14ef3900e9ef449865bb7" is_pre_election: true
I20260621 07:01:50.287488 19338 raft_consensus.cc:2454] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c4a517cd4d48f2a8ef677cb58b858a in term 0.
I20260621 07:01:50.287511 19141 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a, fce315c7ca6b4ab481ad80473bc36ef8; no voters:
I20260621 07:01:50.287863 19536 raft_consensus.cc:2781] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:01:50.288036 19536 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:01:50.288163 19536 raft_consensus.cc:3037] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:50.290339 19536 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:50.291066 19536 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 1 election: Requested vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
I20260621 07:01:50.291446 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511e19852b1e4ebe9224adb60d58178a" candidate_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
I20260621 07:01:50.291642 19469 raft_consensus.cc:3037] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:50.291576 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511e19852b1e4ebe9224adb60d58178a" candidate_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f0d6de23f14ef3900e9ef449865bb7"
I20260621 07:01:50.291828 19338 raft_consensus.cc:3037] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:01:50.293793 19469 raft_consensus.cc:2454] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c4a517cd4d48f2a8ef677cb58b858a in term 1.
I20260621 07:01:50.294045 19338 raft_consensus.cc:2454] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c4a517cd4d48f2a8ef677cb58b858a in term 1.
I20260621 07:01:50.294181 19141 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a, fce315c7ca6b4ab481ad80473bc36ef8; no voters:
I20260621 07:01:50.294520 19536 raft_consensus.cc:2781] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:01:50.295007 19536 raft_consensus.cc:686] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 LEADER]: Becoming Leader. State: Replica: 84c4a517cd4d48f2a8ef677cb58b858a, State: Running, Role: LEADER
I20260621 07:01:50.295611 19536 consensus_queue.cc:237] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:01:50.300035 19065 catalog_manager.cc:5495] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193). New cstate: current_term: 1 leader_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } health_report { overall_health: UNKNOWN } } }
I20260621 07:01:50.385850 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.380615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.2.245.193:34101)
0621 07:01:50.380836 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.380841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.380859 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:01:50.380989 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.380993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.381069 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.381193 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:01:50.381503 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.382279 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.383822 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.384218 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.384313 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.384837 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:01:50.385334 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:01:50.385445 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.385559 (+ 114us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.385626 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260621 07:01:50.394239 19338 raft_consensus.cc:1261] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Refusing update from remote peer 84c4a517cd4d48f2a8ef677cb58b858a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:50.394277 19469 raft_consensus.cc:1261] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Refusing update from remote peer 84c4a517cd4d48f2a8ef677cb58b858a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:01:50.395416 19541 consensus_queue.cc:1035] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [LEADER]: Connected to new peer: Peer: permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:50.395834 19536 consensus_queue.cc:1035] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:01:50.408149 19546 mvcc.cc:205] Tried to move back new op lower bound from 7299175671364984832 to 7299175670974541824. Current Snapshot: MvccSnapshot[applied={T|T < 7299175671364984832}]
I20260621 07:01:50.409876 19544 mvcc.cc:205] Tried to move back new op lower bound from 7299175671364984832 to 7299175670974541824. Current Snapshot: MvccSnapshot[applied={T|T < 7299175671364984832 or (T in {7299175671364984832})}]
I20260621 07:01:50.411913 19545 mvcc.cc:205] Tried to move back new op lower bound from 7299175671364984832 to 7299175670974541824. Current Snapshot: MvccSnapshot[applied={T|T < 7299175671364984832}]
I20260621 07:01:50.419754 19065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:50.420022 19065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:01:50.423385 19065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Sending DeleteTablet for 3 replicas of tablet 511e19852b1e4ebe9224adb60d58178a
I20260621 07:01:50.424544 19449 tablet_service.cc:1514] Processing DeleteTablet for tablet 511e19852b1e4ebe9224adb60d58178a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:50 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:58768
I20260621 07:01:50.424602 19186 tablet_service.cc:1514] Processing DeleteTablet for tablet 511e19852b1e4ebe9224adb60d58178a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:50 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:36554
I20260621 07:01:50.424750 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19103
I20260621 07:01:50.425559 19318 tablet_service.cc:1514] Processing DeleteTablet for tablet 511e19852b1e4ebe9224adb60d58178a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:01:50 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:53718
I20260621 07:01:50.447819 19557 tablet_replica.cc:330] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: stopping tablet replica
I20260621 07:01:50.448482 19557 raft_consensus.cc:2229] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:50.449110 19557 raft_consensus.cc:2258] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:50.450016 19555 tablet_replica.cc:330] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: stopping tablet replica
I20260621 07:01:50.452168 19557 ts_tablet_manager.cc:1905] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:50.452736 19555 raft_consensus.cc:2229] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:01:50.453552 19555 raft_consensus.cc:2258] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:50.455476 19554 tablet_replica.cc:330] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: stopping tablet replica
I20260621 07:01:50.455902 19554 raft_consensus.cc:2229] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:01:50.457284 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.425798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.2.245.193:34101)
0621 07:01:50.425974 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.425982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.436934 (+ 10952us) server_negotiation.cc:407] Connection header received
0621 07:01:50.437417 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.437426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.437529 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.437629 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:50.440176 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.441625 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.444147 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.445058 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.445187 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.449330 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:50.449346 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:50.449365 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:50.449404 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:50.453529 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.454462 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.454470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.454473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.454648 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.455184 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.455191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.455196 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.455666 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:50.455995 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.456636 (+ 641us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.456897 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 07:01:50.458068 19555 ts_tablet_manager.cc:1905] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:50.458954 19246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:50.460390 19554 raft_consensus.cc:2258] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:01:50.462961 19554 ts_tablet_manager.cc:1905] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:01:50.465329 19557 ts_tablet_manager.cc:1918] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:50.465510 19557 log.cc:1196] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/511e19852b1e4ebe9224adb60d58178a
I20260621 07:01:50.466013 19557 ts_tablet_manager.cc:1939] T 511e19852b1e4ebe9224adb60d58178a P 88f0d6de23f14ef3900e9ef449865bb7: Deleting consensus metadata
I20260621 07:01:50.467978 19051 catalog_manager.cc:4841] TS 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): tablet 511e19852b1e4ebe9224adb60d58178a (table test-table [id=33b5759ac5b44067a8555549b460c34f]) successfully deleted
I20260621 07:01:50.469326 19554 ts_tablet_manager.cc:1918] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:01:50.469447 19554 log.cc:1196] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/511e19852b1e4ebe9224adb60d58178a
I20260621 07:01:50.469935 19554 ts_tablet_manager.cc:1939] T 511e19852b1e4ebe9224adb60d58178a P fce315c7ca6b4ab481ad80473bc36ef8: Deleting consensus metadata
I20260621 07:01:50.471372 19052 catalog_manager.cc:4841] TS fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): tablet 511e19852b1e4ebe9224adb60d58178a (table test-table [id=33b5759ac5b44067a8555549b460c34f]) successfully deleted
I20260621 07:01:50.557226 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19103
I20260621 07:01:50.576654 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19255
W20260621 07:01:50.576715 19051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:34101 (error 108)
I20260621 07:01:50.578917 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.577996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36566)
0621 07:01:50.578528 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.578665 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":280,"threads_started":1}
W20260621 07:01:50.579216 19051 catalog_manager.cc:4584] TS 84c4a517cd4d48f2a8ef677cb58b858a (127.2.245.193:34101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 511e19852b1e4ebe9224adb60d58178a: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
I20260621 07:01:50.590705 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.577489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.2.245.194:42263)
0621 07:01:50.577705 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.577709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.577820 (+ 111us) server_negotiation.cc:407] Connection header received
0621 07:01:50.577996 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.578002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.578104 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.578298 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:50.580130 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.580880 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.582182 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.582575 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.582673 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.585392 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:50.585405 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:50.585420 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:50.585455 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:50.588210 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.588867 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.588874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.588877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.589011 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.589378 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.589383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.589385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.589791 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:50.590044 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.590278 (+ 234us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.590441 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:01:50.591499 19378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:50.633039 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.632480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36574)
0621 07:01:50.632751 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.632871 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260621 07:01:50.634138 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.633750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36586)
0621 07:01:50.633897 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.633979 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260621 07:01:50.681284 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19255
I20260621 07:01:50.694726 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.694120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36592)
0621 07:01:50.694381 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.694504 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260621 07:01:50.695966 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.695484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36594)
0621 07:01:50.695663 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.695770 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260621 07:01:50.698494 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19386
I20260621 07:01:50.712913 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.699274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.2.245.195:37297)
0621 07:01:50.699478 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0621 07:01:50.699484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:01:50.699530 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:01:50.699808 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:01:50.699815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:01:50.699910 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:01:50.700019 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:01:50.701368 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.702202 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.703653 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:01:50.704046 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:01:50.704191 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:01:50.706858 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:01:50.706871 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:01:50.706888 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:01:50.706919 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:01:50.709795 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.710537 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.710545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.710547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.710686 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:01:50.711286 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:01:50.711292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:01:50.711295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:01:50.711683 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:01:50.711984 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0621 07:01:50.712315 (+ 331us) server_negotiation.cc:299] Negotiation successful
0621 07:01:50.712543 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260621 07:01:50.713562 19509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:50.776906 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.776421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36604)
0621 07:01:50.776667 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.776760 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260621 07:01:50.778103 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:01:50.777639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:34101 (local address 127.0.0.1:36618)
0621 07:01:50.777895 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0621 07:01:50.777966 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260621 07:01:50.800675 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19386
I20260621 07:01:50.817042 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19033
I20260621 07:01:50.818302 19096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:01:50.929630 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19033
I20260621 07:02:10.951491 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--webserver_interface=127.2.245.254
--webserver_port=43049
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:32929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:11.281914 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:11.282212 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:11.282374 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:11.289767 19564 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 07:02:11.289881 19564 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 07:02:11.289955 19564 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 07:02:11.290033 19564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:11.290126 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:11.290271 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:11.290344 19564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:11.290428 19564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:11.300537 19564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:32929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:32929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=43049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:11.301875 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:11.303259 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:11.315338 19569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:11.315254 19570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:11.315302 19572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:11.315963 19564 server_base.cc:1034] running on GCE node
I20260621 07:02:11.316735 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:11.318152 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:11.319398 19564 hybrid_clock.cc:648] HybridClock initialized: now 1782025331319333 us; error 45 us; skew 500 ppm
I20260621 07:02:11.319736 19564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:11 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:11.322865 19564 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:02:11.324532 19564 webserver.cc:460] Webserver started at http://127.2.245.254:43049/ using document root <none> and password file <none>
I20260621 07:02:11.325239 19564 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:11.325376 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:11.330741 19564 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:11.333722 19579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:11.335367 19564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:11.335583 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "40fcd81d82a34620a4281dac2cfb37fe"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "e0651de0fdf27addd2ecda78b1d90ce6"
server_key_iv: "6e21a72db7e8f7222e2a54229178255e"
server_key_version: "encryptionkey@0"
I20260621 07:02:11.336624 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:11.353917 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:11.357967 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:11.358278 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:11.377704 19564 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:32929
I20260621 07:02:11.377733 19630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:32929 every 8 connection(s)
I20260621 07:02:11.379290 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:11.384848 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19564
I20260621 07:02:11.385697 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:34101
--local_ip_for_outbound_sockets=127.2.245.193
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=36335
--webserver_interface=127.2.245.193
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:02:11.386279 19631 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:02:11.390166 19631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap starting.
I20260621 07:02:11.411212 19631 log.cc:824] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:11.427462 19631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:02:11.428032 19631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Bootstrap complete.
I20260621 07:02:11.437167 19631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:02:11.438066 19631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Initialized, Role: FOLLOWER
I20260621 07:02:11.438719 19631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:02:11.438974 19631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:11.439096 19631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:11.439284 19631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 2 FOLLOWER]: Advancing to term 3
I20260621 07:02:11.443334 19631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:02:11.443868 19631 leader_election.cc:302] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40fcd81d82a34620a4281dac2cfb37fe; no voters:
I20260621 07:02:11.444551 19631 leader_election.cc:288] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260621 07:02:11.444695 19635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 3 FOLLOWER]: Leader election won for term 3
I20260621 07:02:11.446986 19631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:11.451963 19635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [term 3 LEADER]: Becoming Leader. State: Replica: 40fcd81d82a34620a4281dac2cfb37fe, State: Running, Role: LEADER
I20260621 07:02:11.452661 19635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } }
I20260621 07:02:11.455693 19636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:02:11.455906 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40fcd81d82a34620a4281dac2cfb37fe. Latest consensus state: current_term: 3 leader_uuid: "40fcd81d82a34620a4281dac2cfb37fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fcd81d82a34620a4281dac2cfb37fe" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 32929 } } }
I20260621 07:02:11.459460 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:11.459452 19636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:11.472185 19645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:11.480798 19645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:11.482818 19645 catalog_manager.cc:1259] Loaded cluster ID: 66a47441c47749478130cbd4f8d6eb2b
I20260621 07:02:11.483055 19645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:11.488530 19645 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:11.490978 19645 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Loaded TSK: 0
I20260621 07:02:11.491245 19645 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 40fcd81d82a34620a4281dac2cfb37fe: Loaded TSK: 1
F20260621 07:02:11.492466 19645 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3eaab069ad google::LogMessage::Fail() at ??:0
@ 0x7f3eaab0988a google::LogMessage::SendToLog() at ??:0
@ 0x7f3eaab064de google::LogMessage::Flush() at ??:0
@ 0x7f3eaab07a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3eae86a7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3eae85973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3ec05351c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3ec0534783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3ec05c02a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3ec05c00c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3ec060a5e9 std::function<>::operator()() at ??:0
@ 0x7f3ec0538cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3ec0536c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3ec05be610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3ec05be439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3ead752345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3ead75591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3ead755799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3ead730eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3eb0a0a6db start_thread at ??:0
@ 0x7f3ea99a061f clone at ??:0
W20260621 07:02:11.818255 19633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:11.818535 19633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:11.818646 19633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:11.825954 19633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:11.826071 19633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:11.826202 19633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:11.835844 19633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:34101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=36335
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:11.837008 19633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:11.838258 19633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:11.850919 19678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:11.850883 19679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:11.851064 19681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:11.851905 19633 server_base.cc:1034] running on GCE node
I20260621 07:02:11.852500 19633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:11.853399 19633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:11.854629 19633 hybrid_clock.cc:648] HybridClock initialized: now 1782025331854572 us; error 41 us; skew 500 ppm
I20260621 07:02:11.855077 19633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:11 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:11.858265 19633 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:02:11.859789 19633 webserver.cc:460] Webserver started at http://127.2.245.193:36335/ using document root <none> and password file <none>
I20260621 07:02:11.860474 19633 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:11.860607 19633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:11.865912 19633 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260621 07:02:11.868799 19688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:11.870061 19633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:11.870247 19633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "84c4a517cd4d48f2a8ef677cb58b858a"
format_stamp: "Formatted at 2026-06-21 07:01:25 on dist-test-slave-9zdv"
server_key: "a91d4168e714a199b48c3a14666224cd"
server_key_iv: "8fd5a8c3dde570fb9e360ca94e257c4a"
server_key_version: "encryptionkey@0"
I20260621 07:02:11.871279 19633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:11.896322 19633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:11.900298 19633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:11.900579 19633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:11.901584 19633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:11.903430 19695 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260621 07:02:11.908919 19633 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 07:02:11.909049 19633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260621 07:02:11.909193 19633 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 07:02:11.912355 19633 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 07:02:11.912467 19633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:11.912779 19695 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Bootstrap starting.
I20260621 07:02:11.921141 19695 log.cc:824] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:11.936208 19695 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:02:11.937130 19695 tablet_bootstrap.cc:492] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Bootstrap complete.
I20260621 07:02:11.937736 19695 ts_tablet_manager.cc:1397] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.005s
I20260621 07:02:11.948035 19695 raft_consensus.cc:348] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:11.949648 19695 raft_consensus.cc:729] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c4a517cd4d48f2a8ef677cb58b858a, State: Initialized, Role: FOLLOWER
I20260621 07:02:11.950393 19695 consensus_queue.cc:260] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:11.952097 19695 ts_tablet_manager.cc:1428] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260621 07:02:11.958773 19633 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:34101
I20260621 07:02:11.958806 19803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:34101 every 8 connection(s)
I20260621 07:02:11.960055 19633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:11.961687 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19633
I20260621 07:02:11.962462 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:42263
--local_ip_for_outbound_sockets=127.2.245.194
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=35905
--webserver_interface=127.2.245.194
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:02:11.970419 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:11.962833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:54825)
0621 07:02:11.969080 (+ 6247us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:11.969382 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5898,"thread_start_us":5602,"threads_started":1}
W20260621 07:02:11.971179 19804 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:32929 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:32929: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
I20260621 07:02:11.972177 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:11.971771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:39283)
0621 07:02:11.971978 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:11.972047 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260621 07:02:11.973169 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:11.972817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:41367)
0621 07:02:11.972956 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:11.973026 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260621 07:02:11.973445 19804 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:02:12.290179 19807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:12.290474 19807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:12.290642 19807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:12.298417 19807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:12.298522 19807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:12.298707 19807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:12.308858 19807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:42263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=35905
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:12.310166 19807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:12.311558 19807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:12.324623 19812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:12.324790 19813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:12.324851 19815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:12.326206 19807 server_base.cc:1034] running on GCE node
I20260621 07:02:12.326864 19807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:12.327862 19807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:12.329101 19807 hybrid_clock.cc:648] HybridClock initialized: now 1782025332329056 us; error 39 us; skew 500 ppm
I20260621 07:02:12.329435 19807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:12 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:12.332613 19807 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:02:12.334193 19807 webserver.cc:460] Webserver started at http://127.2.245.194:35905/ using document root <none> and password file <none>
I20260621 07:02:12.334903 19807 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:12.335091 19807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:12.340459 19807 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:12.343128 19822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.344383 19807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:02:12.344565 19807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "88f0d6de23f14ef3900e9ef449865bb7"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "f2111eb431065ee67a3e3a08c76c3053"
server_key_iv: "846640c949bca2e785a2601d30dd37a1"
server_key_version: "encryptionkey@0"
I20260621 07:02:12.345553 19807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:12.367949 19807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:12.371819 19807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:12.372093 19807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:12.373054 19807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:12.374445 19807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:12.374573 19807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.374670 19807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:12.374727 19807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.407866 19807 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:42263
I20260621 07:02:12.407953 19934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:42263 every 8 connection(s)
I20260621 07:02:12.409511 19807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:12.415378 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.412674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:53923)
0621 07:02:12.413692 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.414092 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":388,"threads_started":1}
W20260621 07:02:12.416270 19935 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:32929 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:32929: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
I20260621 07:02:12.417356 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.416901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:57533)
0621 07:02:12.417088 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.417186 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260621 07:02:12.418045 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19807
I20260621 07:02:12.418983 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:37297
--local_ip_for_outbound_sockets=127.2.245.195
--tserver_master_addrs=127.2.245.254:32929
--webserver_port=37921
--webserver_interface=127.2.245.195
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:02:12.419842 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.419428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:45871)
0621 07:02:12.419574 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.419687 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260621 07:02:12.420306 19935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:02:12.744728 19938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:12.745008 19938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:12.745119 19938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:12.752862 19938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:12.752969 19938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:12.753075 19938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:12.762477 19938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:37297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=37921
--tserver_master_addrs=127.2.245.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:12.763682 19938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:12.764909 19938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:12.777921 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:12.777961 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:12.778009 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:12.778756 19938 server_base.cc:1034] running on GCE node
I20260621 07:02:12.779381 19938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:12.780230 19938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:12.781425 19938 hybrid_clock.cc:648] HybridClock initialized: now 1782025332781389 us; error 36 us; skew 500 ppm
I20260621 07:02:12.781739 19938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:12 dist-test-slave-9zdv krb5kdc[18483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:12.784921 19938 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:02:12.786326 19938 webserver.cc:460] Webserver started at http://127.2.245.195:37921/ using document root <none> and password file <none>
I20260621 07:02:12.787020 19938 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:12.787128 19938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:12.792956 19938 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260621 07:02:12.795578 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.796871 19938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:12.797061 19938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "fce315c7ca6b4ab481ad80473bc36ef8"
format_stamp: "Formatted at 2026-06-21 07:01:26 on dist-test-slave-9zdv"
server_key: "fe0f237f28add408ae6bba25bbd4690b"
server_key_iv: "21cff75ffe522d2ce5e54cf92fe2ca5f"
server_key_version: "encryptionkey@0"
I20260621 07:02:12.798015 19938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:12.821329 19938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:12.825645 19938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:12.825915 19938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:12.826864 19938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:12.828251 19938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:12.828366 19938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.828500 19938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:12.828589 19938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:12.861660 19938 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37297
I20260621 07:02:12.861747 20065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37297 every 8 connection(s)
I20260621 07:02:12.863260 19938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:12.867965 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.866120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:57867)
0621 07:02:12.866814 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.867150 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":265,"threads_started":1}
W20260621 07:02:12.868618 20066 heartbeater.cc:644] Failed to heartbeat to 127.2.245.254:32929 (0 consecutive failures): Network error: Failed to ping master at 127.2.245.254:32929: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
I20260621 07:02:12.869386 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.869060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:40561)
0621 07:02:12.869185 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.869260 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 07:02:12.870149 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.869876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:36705)
0621 07:02:12.869989 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.870056 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 07:02:12.870385 20066 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 07:02:12.873236 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 19938
I20260621 07:02:12.974756 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:12.973970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:58769)
0621 07:02:12.974464 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:12.974548 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":253,"threads_started":1}
I20260621 07:02:13.421697 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.420804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:33953)
0621 07:02:13.421321 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:13.421426 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":251,"threads_started":1}
I20260621 07:02:13.547523 20072 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:13.547886 20072 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:13.549633 20072 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
Jun 21 07:02:13 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
Jun 21 07:02:13 dist-test-slave-9zdv krb5kdc[18483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:02:13.587318 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.550031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37297 (local address 127.2.245.193:36675)
0621 07:02:13.563120 (+ 13089us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:13.563136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:13.563277 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:13.566740 (+ 3463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:13.566753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:13.568027 (+ 1274us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:13.569135 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:13.569171 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.571534 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.571538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:13.574115 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:13.574133 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.575654 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.575667 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:13.576085 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:13.577069 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:13.577111 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:13.580178 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:13.583756 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:13.583760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:13.583762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:13.584191 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:13.584661 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:13.584667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:13.584843 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:13.585630 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:13.585635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:13.585955 (+ 320us) client_negotiation.cc:769] Sending connection context
0621 07:02:13.586229 (+ 274us) client_negotiation.cc:240] Negotiation successful
0621 07:02:13.586790 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12976,"mutex_wait_us":424,"spinlock_wait_cycles":40576,"thread_start_us":11892,"threads_started":1}
I20260621 07:02:13.587355 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.550574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:41463 (local address 127.2.245.194:42263)
0621 07:02:13.561934 (+ 11360us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:13.561954 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:13.562634 (+ 680us) server_negotiation.cc:407] Connection header received
0621 07:02:13.565660 (+ 3026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:13.565669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:13.566111 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:13.566515 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:13.569373 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.571287 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.574296 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.575402 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.575604 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:13.580136 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:13.580147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:13.580179 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:13.580239 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:13.583416 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584342 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:13.584348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:13.584354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:13.584499 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584990 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:13.584993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:13.584995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:13.585466 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:13.585649 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:13.586511 (+ 862us) server_negotiation.cc:299] Negotiation successful
0621 07:02:13.586887 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11105,"thread_start_us":10895,"threads_started":1}
I20260621 07:02:13.587468 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.550492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:36675 (local address 127.2.245.195:37297)
0621 07:02:13.562722 (+ 12230us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:13.562742 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:13.563271 (+ 529us) server_negotiation.cc:407] Connection header received
0621 07:02:13.565543 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:13.565555 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:13.566111 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:13.566515 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:13.569369 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.571287 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.574326 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.575402 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.575638 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:13.580441 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:13.580448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:13.580461 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:13.580497 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:13.583613 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584344 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:13.584350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:13.584356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:13.584504 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584964 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:13.584973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:13.584976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:13.585473 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:13.585655 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:13.586577 (+ 922us) server_negotiation.cc:299] Negotiation successful
0621 07:02:13.587001 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12012,"thread_start_us":11864,"threads_started":1}
I20260621 07:02:13.587461 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.549996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:42263 (local address 127.2.245.193:41463)
0621 07:02:13.562281 (+ 12285us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:13.562345 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:13.563064 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:13.566740 (+ 3676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:13.566753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:13.568027 (+ 1274us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:13.569135 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:13.569168 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.571529 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.571535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:13.574115 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:13.574133 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:13.575725 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:13.575731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:13.576088 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:13.577127 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:13.577145 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:13.579760 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:13.583572 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:13.583582 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:13.583598 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:13.584175 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:13.584625 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:13.584630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:13.584634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:13.584845 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:13.585612 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:13.585620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:13.585955 (+ 335us) client_negotiation.cc:769] Sending connection context
0621 07:02:13.586241 (+ 286us) client_negotiation.cc:240] Negotiation successful
0621 07:02:13.586982 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12094,"mutex_wait_us":761,"spinlock_wait_cycles":55424,"thread_start_us":12827,"threads_started":1}
W20260621 07:02:13.591106 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:13.591665 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:13.591826 19690 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:13.592242 20072 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:13.871748 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.870862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:36671)
0621 07:02:13.871379 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:13.871519 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":302,"threads_started":1}
I20260621 07:02:13.976338 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:13.975836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:34507)
0621 07:02:13.976019 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:13.976146 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260621 07:02:14.423378 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:14.422606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:43175)
0621 07:02:14.423113 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:14.423194 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":243,"threads_started":1}
I20260621 07:02:14.873085 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:14.872492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:55109)
0621 07:02:14.872872 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:14.872946 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":195,"threads_started":1}
I20260621 07:02:14.977869 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:14.977161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:39469)
0621 07:02:14.977597 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:14.977677 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":219,"threads_started":1}
I20260621 07:02:15.425050 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:15.424333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:53715)
0621 07:02:15.424794 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:15.424877 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":240,"threads_started":1}
I20260621 07:02:15.511986 20082 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:15.512193 20082 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:15.512871 20082 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
W20260621 07:02:15.513893 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:15.514091 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:15.514183 19690 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:15.514408 20082 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:15.874723 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:15.873957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:50305)
0621 07:02:15.874482 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:15.874565 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":302,"threads_started":1}
I20260621 07:02:15.979619 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:15.978824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:46143)
0621 07:02:15.979353 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:15.979430 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
I20260621 07:02:16.426659 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:16.425891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:54087)
0621 07:02:16.426402 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:16.426482 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":280,"threads_started":1}
I20260621 07:02:16.876492 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:16.875747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:35077)
0621 07:02:16.876220 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:16.876307 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":265,"threads_started":1}
I20260621 07:02:16.981294 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:16.980531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:42151)
0621 07:02:16.981001 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:16.981095 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":224,"threads_started":1}
I20260621 07:02:17.428865 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:17.427807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:48497)
0621 07:02:17.428456 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:17.428581 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":319,"threads_started":1}
I20260621 07:02:17.674897 20089 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:17.675158 20089 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:17.675938 20089 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
W20260621 07:02:17.676865 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:17.677037 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:17.677205 19690 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:17.677489 20089 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:17.877978 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:17.877328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:38525)
0621 07:02:17.877744 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:17.877824 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":223,"threads_started":1}
I20260621 07:02:17.983153 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:17.982215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:39163)
0621 07:02:17.982814 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:17.982904 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":346,"threads_started":1}
I20260621 07:02:18.430508 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:18.429774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:60329)
0621 07:02:18.430235 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:18.430325 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260621 07:02:18.879631 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:18.878880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:33711)
0621 07:02:18.879381 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:18.879461 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":297,"threads_started":1}
I20260621 07:02:18.984804 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:18.983931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:47489)
0621 07:02:18.984466 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:18.984555 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":299,"threads_started":1}
I20260621 07:02:19.432338 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:19.431557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:39439)
0621 07:02:19.432068 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:19.432159 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":245,"threads_started":1}
I20260621 07:02:19.881408 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:19.880591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:36243)
0621 07:02:19.881127 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:19.881211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":282,"threads_started":1}
I20260621 07:02:19.986532 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:19.985692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:51775)
0621 07:02:19.986244 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:19.986381 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":266,"threads_started":1}
I20260621 07:02:20.433988 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:20.433264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:42817)
0621 07:02:20.433680 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:20.433773 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":220,"threads_started":1}
I20260621 07:02:20.883059 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:20.882249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:34091)
0621 07:02:20.882732 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:20.882825 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":276,"threads_started":1}
I20260621 07:02:20.988329 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:20.987568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:58587)
0621 07:02:20.988032 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:20.988121 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":287,"threads_started":1}
I20260621 07:02:21.435698 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:21.434916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:48913)
0621 07:02:21.435450 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:21.435540 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":220,"threads_started":1}
I20260621 07:02:21.884624 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:21.883907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:57149)
0621 07:02:21.884370 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:21.884458 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":244,"threads_started":1}
I20260621 07:02:21.989964 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:21.989279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:53723)
0621 07:02:21.989711 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:21.989790 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260621 07:02:22.173808 20104 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:22.174054 20104 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:22.174865 20104 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
W20260621 07:02:22.175849 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:22.176077 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:22.176261 19691 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:22.176585 20104 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:22.440448 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:22.439574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:59417)
0621 07:02:22.440080 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:22.440186 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":292,"threads_started":1}
I20260621 07:02:22.886179 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:22.885489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:36571)
0621 07:02:22.885933 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:22.886011 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":222,"threads_started":1}
I20260621 07:02:22.991535 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:22.990806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:60979)
0621 07:02:22.991262 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:22.991347 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":266,"threads_started":1}
I20260621 07:02:23.442210 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:23.441532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:58103)
0621 07:02:23.441973 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:23.442049 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":214,"threads_started":1}
I20260621 07:02:23.887919 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:23.887171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:45009)
0621 07:02:23.887672 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:23.887760 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":268,"threads_started":1}
I20260621 07:02:23.993075 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:23.992402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:46375)
0621 07:02:23.992816 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:23.992901 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":221,"threads_started":1}
I20260621 07:02:24.443975 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:24.443243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:43139)
0621 07:02:24.443716 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:24.443797 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":238,"threads_started":1}
I20260621 07:02:24.889469 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:24.888759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:43103)
0621 07:02:24.889209 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:24.889295 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
I20260621 07:02:24.994778 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:24.993990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:54425)
0621 07:02:24.994512 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:24.994596 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":267,"threads_started":1}
I20260621 07:02:25.445734 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:25.444937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:56255)
0621 07:02:25.445474 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:25.445570 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260621 07:02:25.891227 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:25.890411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:49701)
0621 07:02:25.890899 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:25.891068 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260621 07:02:25.996488 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:25.995713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:56237)
0621 07:02:25.996163 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:25.996247 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":245,"threads_started":1}
I20260621 07:02:26.094681 20117 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:26.094947 20117 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:26.095818 20117 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
W20260621 07:02:26.096726 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:26.096906 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:26.097069 19690 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:26.097321 20117 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:26.447572 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:26.446752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:39461)
0621 07:02:26.447273 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:26.447348 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":303,"threads_started":1}
I20260621 07:02:26.893028 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:26.892171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:56147)
0621 07:02:26.892720 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:26.892826 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":308,"threads_started":1}
I20260621 07:02:26.998639 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:26.997853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.193:58281)
0621 07:02:26.998384 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:26.998470 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":305,"threads_started":1}
I20260621 07:02:27.449980 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:27.448978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.194:53735)
0621 07:02:27.449633 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:27.449802 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":288,"threads_started":1}
I20260621 07:02:27.712605 20122 raft_consensus.cc:482] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:27.712788 20122 raft_consensus.cc:504] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce315c7ca6b4ab481ad80473bc36ef8" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37297 } } peers { permanent_uuid: "84c4a517cd4d48f2a8ef677cb58b858a" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 34101 } } peers { permanent_uuid: "88f0d6de23f14ef3900e9ef449865bb7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 42263 } }
I20260621 07:02:27.713402 20122 leader_election.cc:288] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297), 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263)
W20260621 07:02:27.714588 19691 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fce315c7ca6b4ab481ad80473bc36ef8 (127.2.245.195:37297): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
W20260621 07:02:27.714800 19690 leader_election.cc:341] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88f0d6de23f14ef3900e9ef449865bb7 (127.2.245.194:42263): Not found: Tablet not found: 511e19852b1e4ebe9224adb60d58178a
I20260621 07:02:27.714998 19690 leader_election.cc:302] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84c4a517cd4d48f2a8ef677cb58b858a; no voters: 88f0d6de23f14ef3900e9ef449865bb7, fce315c7ca6b4ab481ad80473bc36ef8
I20260621 07:02:27.715281 20122 raft_consensus.cc:2726] T 511e19852b1e4ebe9224adb60d58178a P 84c4a517cd4d48f2a8ef677cb58b858a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:02:27.880560 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19633
I20260621 07:02:27.904225 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:27.881601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.2.245.193:34101)
0621 07:02:27.882343 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:27.882355 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:27.882393 (+ 38us) server_negotiation.cc:407] Connection header received
0621 07:02:27.882473 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:27.882484 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:27.882782 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:27.883020 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:27.884401 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:27.885377 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:27.886619 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:27.887290 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:27.887439 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:27.895707 (+ 8268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:27.895729 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:27.895752 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:27.895809 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:27.900335 (+ 4526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:27.901274 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:27.901283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:27.901287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:27.901507 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:27.901994 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:27.902026 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:27.902032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:27.902651 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:27.903076 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:27.903360 (+ 284us) server_negotiation.cc:299] Negotiation successful
0621 07:02:27.903639 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":310,"threads_started":1}
I20260621 07:02:27.905716 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:27.899799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:32929 (local address 127.2.245.195:38995)
0621 07:02:27.905304 (+ 5505us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:27.905484 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.254:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5346,"thread_start_us":5192,"threads_started":1}
I20260621 07:02:27.906070 19799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:28.012723 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19633
I20260621 07:02:28.031724 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19807
I20260621 07:02:28.045473 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:28.032596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.2.245.194:42263)
0621 07:02:28.033187 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:28.033191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:28.033207 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:02:28.033262 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:28.033265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:28.033325 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:28.033443 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:28.034753 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:28.035615 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:28.036919 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:28.037339 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:28.037472 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:28.040117 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:28.040122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:28.040125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:28.040158 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:28.042741 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:28.043442 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:28.043448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:28.043451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:28.043574 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:28.043944 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:28.043948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:28.043971 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:28.044241 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:28.044520 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:28.044842 (+ 322us) server_negotiation.cc:299] Negotiation successful
0621 07:02:28.045089 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":319,"threads_started":1}
I20260621 07:02:28.046250 19930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:28.131429 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19807
I20260621 07:02:28.149814 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 19938
I20260621 07:02:28.163537 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:28.150750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.2.245.195:37297)
0621 07:02:28.151289 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:28.151293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:28.151312 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:02:28.151403 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:28.151407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:28.151475 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:28.151570 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:28.152904 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:28.153663 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:28.154887 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:28.155671 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:28.155781 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:28.158364 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:28.158369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:28.158372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:28.158402 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:28.160983 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:28.161705 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:28.161709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:28.161711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:28.161854 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:28.162182 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:28.162185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:28.162187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:28.162432 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:28.162628 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:28.163093 (+ 465us) server_negotiation.cc:299] Negotiation successful
0621 07:02:28.163254 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":284,"threads_started":1}
I20260621 07:02:28.164225 20061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:28.248755 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 19938
2026-06-21T07:02:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64600 ms)
[----------] 30 tests from SecurityITest (291046 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:28 dist-test-slave-9zdv krb5kdc[20133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:28 dist-test-slave-9zdv krb5kdc[20133](info): set up 2 sockets
Jun 21 07:02:28 dist-test-slave-9zdv krb5kdc[20133](info): commencing operation
krb5kdc: starting...
W20260621 07:02:29.351440 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:29 dist-test-slave-9zdv krb5kdc[20133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:02:29.612812 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:42665
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:42665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:29.949738 20146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:29.950104 20146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:29.950278 20146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:29.957737 20146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:29.957906 20146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:29.958021 20146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:29.958119 20146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:29.958215 20146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:29.968511 20146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:42665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:42665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:29.969942 20146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:29.971381 20146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:29.984028 20152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:29.984313 20151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:29.984028 20154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:29.984689 20146 server_base.cc:1034] running on GCE node
I20260621 07:02:29.985538 20146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:29.987098 20146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:29.988305 20146 hybrid_clock.cc:648] HybridClock initialized: now 1782025349988267 us; error 37 us; skew 500 ppm
I20260621 07:02:29.988767 20146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:29 dist-test-slave-9zdv krb5kdc[20133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:29.992074 20146 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:02:29.993757 20146 webserver.cc:460] Webserver started at http://127.2.245.254:43147/ using document root <none> and password file <none>
I20260621 07:02:29.994470 20146 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:29.994621 20146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:29.994983 20146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:29.998106 20146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "8138a71f22db4ade894dfa2d2b9febc1"
format_stamp: "Formatted at 2026-06-21 07:02:29 on dist-test-slave-9zdv"
server_key: "4c8e14ab5a9acacc210bf156a18bac2e"
server_key_iv: "41640c87da0cdc5fe62d3133da50e02c"
server_key_version: "encryptionkey@0"
I20260621 07:02:29.998929 20146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "8138a71f22db4ade894dfa2d2b9febc1"
format_stamp: "Formatted at 2026-06-21 07:02:29 on dist-test-slave-9zdv"
server_key: "4c8e14ab5a9acacc210bf156a18bac2e"
server_key_iv: "41640c87da0cdc5fe62d3133da50e02c"
server_key_version: "encryptionkey@0"
I20260621 07:02:30.004505 20146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 07:02:30.008162 20161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:30.009546 20146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:30.009742 20146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "8138a71f22db4ade894dfa2d2b9febc1"
format_stamp: "Formatted at 2026-06-21 07:02:29 on dist-test-slave-9zdv"
server_key: "4c8e14ab5a9acacc210bf156a18bac2e"
server_key_iv: "41640c87da0cdc5fe62d3133da50e02c"
server_key_version: "encryptionkey@0"
I20260621 07:02:30.009955 20146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:30.030102 20146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:30.034575 20146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:30.034907 20146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:30.053993 20146 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:42665
I20260621 07:02:30.054015 20212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:42665 every 8 connection(s)
I20260621 07:02:30.055452 20146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:30.057509 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20146
I20260621 07:02:30.057770 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:30.058394 3031 external_mini_cluster.cc:1428] Setting key 66a43e8170b0e0e60b21db7c8ba18604
I20260621 07:02:30.059458 20213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:30.071745 20213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: Bootstrap starting.
Jun 21 07:02:30 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:30.075594 20213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:30.076683 20213 log.cc:824] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:30.080251 20213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: No bootstrap required, opened a new log
I20260621 07:02:30.081121 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:30.060416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57622 (local address 127.2.245.254:42665)
0621 07:02:30.061168 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:30.061179 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:30.061217 (+ 38us) server_negotiation.cc:407] Connection header received
0621 07:02:30.062845 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:30.062866 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:30.063335 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:30.063789 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:30.065301 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.066628 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.068000 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.068712 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.068918 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:30.073517 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:30.073524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:30.073539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:30.073579 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:30.076655 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:30.077590 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:30.077617 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:30.077627 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:30.077842 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:30.078415 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:30.078418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:30.078420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:30.078930 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:30.079219 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:30.079686 (+ 467us) server_negotiation.cc:299] Negotiation successful
0621 07:02:30.080046 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":336,"threads_started":1}
I20260621 07:02:30.087872 20213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } }
I20260621 07:02:30.088245 20213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:30.088376 20213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8138a71f22db4ade894dfa2d2b9febc1, State: Initialized, Role: FOLLOWER
I20260621 07:02:30.089017 20213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } }
I20260621 07:02:30.089252 20213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:30.089385 20213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:30.089594 20213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:30.091986 20213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } }
I20260621 07:02:30.092501 20213 leader_election.cc:302] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8138a71f22db4ade894dfa2d2b9febc1; no voters:
I20260621 07:02:30.093080 20213 leader_election.cc:288] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:30.093223 20218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:30.093899 20218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [term 1 LEADER]: Becoming Leader. State: Replica: 8138a71f22db4ade894dfa2d2b9febc1, State: Running, Role: LEADER
I20260621 07:02:30.094484 20218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } }
I20260621 07:02:30.095331 20213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:30.097559 20219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8138a71f22db4ade894dfa2d2b9febc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } } }
I20260621 07:02:30.097932 20219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:30.097664 20220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8138a71f22db4ade894dfa2d2b9febc1. Latest consensus state: current_term: 1 leader_uuid: "8138a71f22db4ade894dfa2d2b9febc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8138a71f22db4ade894dfa2d2b9febc1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 42665 } } }
I20260621 07:02:30.098320 20220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:30.099646 20224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:30.105019 20224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:30.114719 20224 catalog_manager.cc:1347] Generated new cluster ID: b99d932374644bdba8a99b7a260752c6
I20260621 07:02:30.114845 20224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:30.130419 20224 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:30.132259 20224 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:30.141637 20224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: Generated new TSK 0
I20260621 07:02:30.142473 20224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:02:30.210708 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:30.595669 20239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:30.595953 20239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:30.596165 20239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:30.603564 20239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:30.603667 20239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:30.603785 20239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:30.613461 20239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:30.614820 20239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:30.616127 20239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:30.629216 20244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:30.629235 20245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:30.629760 20247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:30.630062 20239 server_base.cc:1034] running on GCE node
I20260621 07:02:30.630704 20239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:30.631644 20239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:30.632863 20239 hybrid_clock.cc:648] HybridClock initialized: now 1782025350632790 us; error 29 us; skew 500 ppm
I20260621 07:02:30.633322 20239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:30 dist-test-slave-9zdv krb5kdc[20133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025350, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:30.636772 20239 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:02:30.638199 20239 webserver.cc:460] Webserver started at http://127.2.245.193:32973/ using document root <none> and password file <none>
I20260621 07:02:30.638851 20239 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:30.639014 20239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:30.639338 20239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:30.642233 20239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "be1b7f5b0af94c66a9387efaff3f62b3"
format_stamp: "Formatted at 2026-06-21 07:02:30 on dist-test-slave-9zdv"
server_key: "ae8f41b3f6007a780871dd2c3cf3dbcf"
server_key_iv: "9dd43b5241bb26c9ccc8153893c501a3"
server_key_version: "encryptionkey@0"
I20260621 07:02:30.643062 20239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "be1b7f5b0af94c66a9387efaff3f62b3"
format_stamp: "Formatted at 2026-06-21 07:02:30 on dist-test-slave-9zdv"
server_key: "ae8f41b3f6007a780871dd2c3cf3dbcf"
server_key_iv: "9dd43b5241bb26c9ccc8153893c501a3"
server_key_version: "encryptionkey@0"
I20260621 07:02:30.648353 20239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 07:02:30.651815 20254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:30.653178 20239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:30.653362 20239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "be1b7f5b0af94c66a9387efaff3f62b3"
format_stamp: "Formatted at 2026-06-21 07:02:30 on dist-test-slave-9zdv"
server_key: "ae8f41b3f6007a780871dd2c3cf3dbcf"
server_key_iv: "9dd43b5241bb26c9ccc8153893c501a3"
server_key_version: "encryptionkey@0"
I20260621 07:02:30.653654 20239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:30.672034 20239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:30.676080 20239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:30.676383 20239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:30.677368 20239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:30.678728 20239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:30.678804 20239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:30.678925 20239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:30.679018 20239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260621 07:02:30.712745 20239 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:35643
I20260621 07:02:30.712846 20366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:35643 every 8 connection(s)
I20260621 07:02:30.714202 20239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:30.720551 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20239
I20260621 07:02:30.720785 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:30.721647 3031 external_mini_cluster.cc:1428] Setting key 84a56b99dc2a5052225bf70616d9f1e5
Jun 21 07:02:30 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025350, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:30.737263 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:30.716813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:37099 (local address 127.2.245.254:42665)
0621 07:02:30.717295 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:30.717299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:30.718252 (+ 953us) server_negotiation.cc:407] Connection header received
0621 07:02:30.720235 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:30.720240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:30.720308 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:30.720395 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:30.722537 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.723633 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.726120 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.726498 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.726592 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:30.730873 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:30.730878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:30.730881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:30.730913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:30.734105 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:30.734835 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:30.734840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:30.734844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:30.735033 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:30.735475 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:30.735480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:30.735484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:30.735769 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:30.735921 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:30.736625 (+ 704us) server_negotiation.cc:299] Negotiation successful
0621 07:02:30.736896 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260621 07:02:30.738241 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:30.717418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42665 (local address 127.2.245.193:37099)
0621 07:02:30.718090 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:30.718134 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:30.719887 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:30.720707 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:30.720720 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:30.721519 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:30.722348 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:30.722369 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.723814 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.723819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:30.725957 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:30.725976 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:30.726737 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:30.726746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:30.727120 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:30.727879 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:30.727919 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:30.730641 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:30.734245 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:30.734254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:30.734271 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:30.734698 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:30.735150 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:30.735155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:30.735159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:30.735356 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:30.735908 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:30.735916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:30.736268 (+ 352us) client_negotiation.cc:769] Sending connection context
0621 07:02:30.736544 (+ 276us) client_negotiation.cc:240] Negotiation successful
0621 07:02:30.737099 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":257,"threads_started":1}
I20260621 07:02:30.739835 20367 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42665
I20260621 07:02:30.740162 20367 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:30.741050 20367 heartbeater.cc:511] Master 127.2.245.254:42665 requested a full tablet report, sending...
I20260621 07:02:30.743326 20178 ts_manager.cc:194] Registered new tserver with Master: be1b7f5b0af94c66a9387efaff3f62b3 (127.2.245.193:35643)
I20260621 07:02:30.745175 20178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:37099
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 07:02:30.778559 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:31.113224 20373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:31.113480 20373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:31.113616 20373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:31.120786 20373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:31.120887 20373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:31.121012 20373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:31.130366 20373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:31.131582 20373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:31.132830 20373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:31.145780 20378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:31.145820 20379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:31.145839 20381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:31.146592 20373 server_base.cc:1034] running on GCE node
I20260621 07:02:31.147264 20373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:31.148202 20373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:31.149434 20373 hybrid_clock.cc:648] HybridClock initialized: now 1782025351149370 us; error 52 us; skew 500 ppm
I20260621 07:02:31.149772 20373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:31 dist-test-slave-9zdv krb5kdc[20133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:31.153116 20373 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:02:31.154613 20373 webserver.cc:460] Webserver started at http://127.2.245.194:36147/ using document root <none> and password file <none>
I20260621 07:02:31.155367 20373 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:31.155507 20373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:31.155820 20373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:31.158751 20373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "b211485a348d465ca4f5c9a3fa2357bb"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "6821247ac7440fa0aaee3c9226832470"
server_key_iv: "5b1aac0606ad8858d8f650c878f631ae"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.159610 20373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "b211485a348d465ca4f5c9a3fa2357bb"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "6821247ac7440fa0aaee3c9226832470"
server_key_iv: "5b1aac0606ad8858d8f650c878f631ae"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.164600 20373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:02:31.167968 20388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.169207 20373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:31.169370 20373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "b211485a348d465ca4f5c9a3fa2357bb"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "6821247ac7440fa0aaee3c9226832470"
server_key_iv: "5b1aac0606ad8858d8f650c878f631ae"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.169569 20373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:31.181541 20373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:31.185879 20373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:31.186173 20373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:31.187180 20373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:31.188496 20373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:31.188598 20373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.188741 20373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:31.188812 20373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.221891 20373 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:36321
I20260621 07:02:31.221935 20500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:36321 every 8 connection(s)
I20260621 07:02:31.223237 20373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:31.232550 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20373
I20260621 07:02:31.232743 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:31.233431 3031 external_mini_cluster.cc:1428] Setting key 420b0e50ed6e258a80c416b80ca90e5a
Jun 21 07:02:31 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:31.245918 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.225885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:48257 (local address 127.2.245.254:42665)
0621 07:02:31.226068 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.226072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.227290 (+ 1218us) server_negotiation.cc:407] Connection header received
0621 07:02:31.229109 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.229113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.229184 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.229299 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:31.230900 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.231902 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.234356 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.234984 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.235114 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.239329 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:31.239334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:31.239338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:31.239374 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:31.242170 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.243201 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.243205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.243208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.243364 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.243906 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.243913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.243918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.244318 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:31.244483 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.245198 (+ 715us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.245467 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260621 07:02:31.246596 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.226386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42665 (local address 127.2.245.194:48257)
0621 07:02:31.227131 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:31.227170 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:31.228834 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:31.229534 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:31.229543 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:31.230120 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:31.230704 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.230722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.232076 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.232080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.234194 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.234208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.235202 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.235207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.235512 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.236250 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:31.236273 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:31.239037 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:31.242331 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:31.242343 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:31.242364 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:31.243070 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:31.243488 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:31.243493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:31.243498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:31.243763 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:31.244457 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:31.244464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:31.244811 (+ 347us) client_negotiation.cc:769] Sending connection context
0621 07:02:31.245043 (+ 232us) client_negotiation.cc:240] Negotiation successful
0621 07:02:31.245442 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":287,"threads_started":1}
I20260621 07:02:31.248133 20501 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42665
I20260621 07:02:31.248456 20501 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:31.249307 20501 heartbeater.cc:511] Master 127.2.245.254:42665 requested a full tablet report, sending...
I20260621 07:02:31.250749 20178 ts_manager.cc:194] Registered new tserver with Master: b211485a348d465ca4f5c9a3fa2357bb (127.2.245.194:36321)
I20260621 07:02:31.251603 20178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:48257
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:02:31.290127 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:31.618424 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:31.618712 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:31.618887 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:31.626376 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:31.626479 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:31.626682 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:31.636379 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:31.637599 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:31.639150 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:31.652280 20512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:31.652372 20511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:31.652280 20514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:31.653126 20506 server_base.cc:1034] running on GCE node
I20260621 07:02:31.653827 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:31.654858 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:31.656075 20506 hybrid_clock.cc:648] HybridClock initialized: now 1782025351656031 us; error 37 us; skew 500 ppm
I20260621 07:02:31.656425 20506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:31 dist-test-slave-9zdv krb5kdc[20133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:31.659775 20506 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:02:31.661180 20506 webserver.cc:460] Webserver started at http://127.2.245.195:41299/ using document root <none> and password file <none>
I20260621 07:02:31.662142 20506 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:31.662292 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:31.662744 20506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:31.665774 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "68c3c51f76c64ffb849e00de209e1e78"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "2aa5a4fa3291ae8bc2ddbfebbeca5079"
server_key_iv: "423dee651b384490b49466e98cf89f93"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.666486 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "68c3c51f76c64ffb849e00de209e1e78"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "2aa5a4fa3291ae8bc2ddbfebbeca5079"
server_key_iv: "423dee651b384490b49466e98cf89f93"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.671557 20506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:02:31.675242 20521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.676697 20506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:02:31.676831 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "68c3c51f76c64ffb849e00de209e1e78"
format_stamp: "Formatted at 2026-06-21 07:02:31 on dist-test-slave-9zdv"
server_key: "2aa5a4fa3291ae8bc2ddbfebbeca5079"
server_key_iv: "423dee651b384490b49466e98cf89f93"
server_key_version: "encryptionkey@0"
I20260621 07:02:31.676999 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:31.690307 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:31.696118 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:31.696362 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:31.697240 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:31.698436 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:31.698504 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.698581 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:31.698630 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:31.733641 20506 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:42353
I20260621 07:02:31.733737 20633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:42353 every 8 connection(s)
I20260621 07:02:31.734928 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:31.743969 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20506
I20260621 07:02:31.744159 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:31.744575 3031 external_mini_cluster.cc:1428] Setting key 008f8ed018bb84a1e8f795c194e07a53
Jun 21 07:02:31 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:31.749357 20367 heartbeater.cc:503] Master 127.2.245.254:42665 was elected leader, sending a full tablet report...
I20260621 07:02:31.755966 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.737572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:35021 (local address 127.2.245.254:42665)
0621 07:02:31.737757 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.737761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.738775 (+ 1014us) server_negotiation.cc:407] Connection header received
0621 07:02:31.740629 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.740637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.740731 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.740844 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:31.742369 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.743141 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.744870 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.745382 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.745510 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.750079 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:31.750085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:31.750089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:31.750132 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:31.753057 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.753747 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.753751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.753753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.753873 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.754233 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.754236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.754238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.754491 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:31.754602 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.755520 (+ 918us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.755727 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:02:31.757077 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.737921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:42665 (local address 127.2.245.195:35021)
0621 07:02:31.738587 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:31.738633 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:31.740322 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:31.741086 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:31.741094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:31.741623 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:31.742181 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.742198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.743310 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.743315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.744730 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.744743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.745680 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.745687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.746018 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.746723 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:31.746745 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:31.749854 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:31.753182 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:31.753188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:31.753201 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:31.753644 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:31.753966 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:31.753969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:31.753972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:31.754144 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:31.754643 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:31.754650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:31.755110 (+ 460us) client_negotiation.cc:769] Sending connection context
0621 07:02:31.755416 (+ 306us) client_negotiation.cc:240] Negotiation successful
0621 07:02:31.755853 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":263,"threads_started":1}
I20260621 07:02:31.758621 20634 heartbeater.cc:347] Connected to a master server at 127.2.245.254:42665
I20260621 07:02:31.758984 20634 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:31.759666 20634 heartbeater.cc:511] Master 127.2.245.254:42665 requested a full tablet report, sending...
I20260621 07:02:31.761116 20178 ts_manager.cc:194] Registered new tserver with Master: 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353)
I20260621 07:02:31.761864 20178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:35021
I20260621 07:02:31.773109 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:31.790309 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.777695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57632 (local address 127.2.245.254:42665)
0621 07:02:31.777862 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.777867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.778305 (+ 438us) server_negotiation.cc:407] Connection header received
0621 07:02:31.778439 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.778443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.778520 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.778622 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:31.780165 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.780991 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.782425 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.782838 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.782992 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.785328 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:31.785333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:31.785336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:31.785368 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:31.787897 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.788541 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.788545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.788547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.788683 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:31.789008 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:31.789011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:31.789013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:31.789370 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:31.789562 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.789815 (+ 253us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.789973 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:02:31.794323 20178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:31.801256 20178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:31.824651 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.814321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.2.245.195:42353)
0621 07:02:31.815998 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.816007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.816044 (+ 37us) server_negotiation.cc:407] Connection header received
0621 07:02:31.816164 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.816173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.816454 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.817366 (+ 912us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:31.817649 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.819012 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.821874 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.823395 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.823525 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.823816 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.824104 (+ 288us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.824274 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1460,"thread_start_us":1236,"threads_started":1}
I20260621 07:02:31.824743 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.813556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:42353 (local address 127.0.0.1:56242)
0621 07:02:31.815186 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:31.815220 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:31.815337 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:31.816726 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:31.816734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:31.816764 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:31.817175 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.817191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.819154 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.819160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.821704 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.821720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.823604 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.823613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.823813 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.823877 (+ 64us) client_negotiation.cc:769] Sending connection context
0621 07:02:31.824028 (+ 151us) client_negotiation.cc:240] Negotiation successful
0621 07:02:31.824232 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1473,"mutex_wait_us":584,"thread_start_us":436,"threads_started":1}
I20260621 07:02:31.825482 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.815525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38030)
0621 07:02:31.816289 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:31.816310 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:31.816478 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:31.817666 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:31.817674 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:31.817702 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:31.818224 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.818236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.820223 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.820227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.822670 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.822686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.824545 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.824552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.824764 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.824811 (+ 47us) client_negotiation.cc:769] Sending connection context
0621 07:02:31.824883 (+ 72us) client_negotiation.cc:240] Negotiation successful
0621 07:02:31.825014 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":101,"thread_start_us":440,"threads_started":1}
I20260621 07:02:31.825733 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.813502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.2.245.194:36321)
0621 07:02:31.816639 (+ 3137us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.816649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.816679 (+ 30us) server_negotiation.cc:407] Connection header received
0621 07:02:31.816765 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.816771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.817011 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.818172 (+ 1161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:31.818499 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.820004 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.822869 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.824136 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.824914 (+ 778us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.825135 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.825207 (+ 72us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.825391 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2946,"thread_start_us":2807,"threads_started":1}
I20260621 07:02:31.826679 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.815907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.2.245.193:35643)
0621 07:02:31.817066 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:31.817072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:31.817143 (+ 71us) server_negotiation.cc:407] Connection header received
0621 07:02:31.817240 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:31.817246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:31.817461 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:31.817830 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:31.818398 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.820005 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.822872 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.824287 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.825846 (+ 1559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.826137 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:31.826203 (+ 66us) server_negotiation.cc:299] Negotiation successful
0621 07:02:31.826359 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":989,"thread_start_us":841,"threads_started":1}
I20260621 07:02:31.826968 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:31.813265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36321 (local address 127.0.0.1:39320)
0621 07:02:31.814374 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:31.814418 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:31.814589 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:31.817253 (+ 2664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:31.817258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:31.817283 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:31.817723 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.817738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.820148 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.820153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.822676 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:31.822690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:31.824366 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:31.824374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:31.824547 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:31.824587 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 07:02:31.824659 (+ 72us) client_negotiation.cc:240] Negotiation successful
0621 07:02:31.824807 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":370,"thread_start_us":1833,"threads_started":1}
I20260621 07:02:31.829810 20436 tablet_service.cc:1467] Processing CreateTablet for tablet 6520b679a3694c678e76981ebc226452 (DEFAULT_TABLE table=test-table [id=f5d008412b79450ca1817ae8e33ebb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:31.829589 20569 tablet_service.cc:1467] Processing CreateTablet for tablet 6520b679a3694c678e76981ebc226452 (DEFAULT_TABLE table=test-table [id=f5d008412b79450ca1817ae8e33ebb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:31.830154 20302 tablet_service.cc:1467] Processing CreateTablet for tablet 6520b679a3694c678e76981ebc226452 (DEFAULT_TABLE table=test-table [id=f5d008412b79450ca1817ae8e33ebb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:31.831210 20436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6520b679a3694c678e76981ebc226452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:31.831238 20569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6520b679a3694c678e76981ebc226452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:31.831506 20302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6520b679a3694c678e76981ebc226452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:31.842681 20652 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Bootstrap starting.
I20260621 07:02:31.845736 20653 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Bootstrap starting.
I20260621 07:02:31.846276 20652 tablet_bootstrap.cc:654] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:31.847721 20652 log.cc:824] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:31.849640 20653 tablet_bootstrap.cc:654] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:31.851253 20653 log.cc:824] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:31.851766 20654 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Bootstrap starting.
I20260621 07:02:31.854595 20654 tablet_bootstrap.cc:654] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:31.855979 20654 log.cc:824] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:31.857713 20652 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: No bootstrap required, opened a new log
I20260621 07:02:31.857919 20653 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: No bootstrap required, opened a new log
I20260621 07:02:31.858012 20652 ts_tablet_manager.cc:1397] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260621 07:02:31.858180 20653 ts_tablet_manager.cc:1397] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260621 07:02:31.859163 20654 tablet_bootstrap.cc:492] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: No bootstrap required, opened a new log
I20260621 07:02:31.859407 20654 ts_tablet_manager.cc:1397] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260621 07:02:31.865445 20653 raft_consensus.cc:348] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.865856 20653 raft_consensus.cc:374] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:31.865950 20653 raft_consensus.cc:729] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c3c51f76c64ffb849e00de209e1e78, State: Initialized, Role: FOLLOWER
I20260621 07:02:31.866578 20653 consensus_queue.cc:260] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.868472 20653 ts_tablet_manager.cc:1428] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260621 07:02:31.868953 20634 heartbeater.cc:503] Master 127.2.245.254:42665 was elected leader, sending a full tablet report...
I20260621 07:02:31.869045 20652 raft_consensus.cc:348] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.869467 20652 raft_consensus.cc:374] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:31.869553 20652 raft_consensus.cc:729] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b211485a348d465ca4f5c9a3fa2357bb, State: Initialized, Role: FOLLOWER
I20260621 07:02:31.870023 20654 raft_consensus.cc:348] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.870162 20652 consensus_queue.cc:260] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.870467 20654 raft_consensus.cc:374] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:31.870558 20654 raft_consensus.cc:729] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be1b7f5b0af94c66a9387efaff3f62b3, State: Initialized, Role: FOLLOWER
I20260621 07:02:31.871276 20654 consensus_queue.cc:260] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:31.872051 20652 ts_tablet_manager.cc:1428] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260621 07:02:31.872912 20654 ts_tablet_manager.cc:1428] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260621 07:02:31.872946 20501 heartbeater.cc:503] Master 127.2.245.254:42665 was elected leader, sending a full tablet report...
W20260621 07:02:31.966611 20368 tablet.cc:2369] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:31.975255 20502 tablet.cc:2369] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:31.986532 20635 tablet.cc:2369] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:32.198784 20660 raft_consensus.cc:482] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:32.199151 20660 raft_consensus.cc:504] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:32.199884 20659 raft_consensus.cc:482] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:32.200212 20659 raft_consensus.cc:504] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:32.201067 20660 leader_election.cc:288] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b211485a348d465ca4f5c9a3fa2357bb (127.2.245.194:36321), 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353)
I20260621 07:02:32.202157 20659 leader_election.cc:288] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353), be1b7f5b0af94c66a9387efaff3f62b3 (127.2.245.193:35643)
I20260621 07:02:32.211652 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.201643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:56161 (local address 127.2.245.195:42353)
0621 07:02:32.202098 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.202104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.203324 (+ 1220us) server_negotiation.cc:407] Connection header received
0621 07:02:32.203554 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.203571 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.203652 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.203713 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:32.204601 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.205742 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.209178 (+ 3436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.210414 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.210553 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.210626 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.211223 (+ 597us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.211357 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260621 07:02:32.211871 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.201989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:49279 (local address 127.2.245.194:36321)
0621 07:02:32.202206 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.202215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.202905 (+ 690us) server_negotiation.cc:407] Connection header received
0621 07:02:32.203279 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.203287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.203381 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.203458 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:32.204352 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.205775 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.208892 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.210584 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.210724 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.210805 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.211371 (+ 566us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.211506 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":55}
I20260621 07:02:32.212525 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.201706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:42353 (local address 127.2.245.193:56161)
0621 07:02:32.203244 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.203260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:32.203410 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:32.203908 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:32.203912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:32.203930 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:32.204395 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.204407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.205917 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.205923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.208980 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.208995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.210738 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.210744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.210981 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.211044 (+ 63us) client_negotiation.cc:769] Sending connection context
0621 07:02:32.211942 (+ 898us) client_negotiation.cc:240] Negotiation successful
0621 07:02:32.212089 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1415,"mutex_wait_us":444,"thread_start_us":814,"threads_started":1}
I20260621 07:02:32.212766 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.201436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:36321 (local address 127.2.245.193:49279)
0621 07:02:32.202789 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.202810 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:32.203075 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:32.203668 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:32.203673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:32.203708 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:32.204169 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.204183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.205988 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.205994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.208729 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.208745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.210796 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.210802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.211119 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.211171 (+ 52us) client_negotiation.cc:769] Sending connection context
0621 07:02:32.211300 (+ 129us) client_negotiation.cc:240] Negotiation successful
0621 07:02:32.212269 (+ 969us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"mutex_wait_us":983,"thread_start_us":1521,"threads_started":1}
I20260621 07:02:32.213775 20589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c3c51f76c64ffb849e00de209e1e78" is_pre_election: true
I20260621 07:02:32.213898 20456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b211485a348d465ca4f5c9a3fa2357bb" is_pre_election: true
I20260621 07:02:32.214406 20456 raft_consensus.cc:2454] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1b7f5b0af94c66a9387efaff3f62b3 in term 0.
I20260621 07:02:32.214445 20589 raft_consensus.cc:2454] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1b7f5b0af94c66a9387efaff3f62b3 in term 0.
I20260621 07:02:32.214617 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.203628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:42353 (local address 127.2.245.194:35637)
0621 07:02:32.205580 (+ 1952us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.205597 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:32.205712 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:32.206687 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:32.206693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:32.206719 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:32.207169 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.207180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.208710 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.208713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.211528 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.211542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.213529 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.213548 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.213739 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.213779 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 07:02:32.213887 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:02:32.214030 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"mutex_wait_us":393,"thread_start_us":1612,"threads_started":1}
I20260621 07:02:32.215263 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.204143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:35637 (local address 127.2.245.195:42353)
0621 07:02:32.206204 (+ 2061us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.206210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.206232 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:02:32.206284 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.206290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.206356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.206453 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:32.207356 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.208534 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.211722 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.213090 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.213247 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.213321 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.213935 (+ 614us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.214317 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1933,"thread_start_us":1789,"threads_started":1}
I20260621 07:02:32.215251 20256 leader_election.cc:302] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b211485a348d465ca4f5c9a3fa2357bb, be1b7f5b0af94c66a9387efaff3f62b3; no voters:
I20260621 07:02:32.215679 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.204445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:58585 (local address 127.2.245.193:35643)
0621 07:02:32.204621 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.204626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.204641 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:02:32.204695 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.204700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.204760 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.204840 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:32.206689 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.207950 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.211007 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.212313 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.212477 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.212572 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.215270 (+ 2698us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.215437 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":59}
I20260621 07:02:32.215741 20660 raft_consensus.cc:2781] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:32.215798 20589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "b211485a348d465ca4f5c9a3fa2357bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c3c51f76c64ffb849e00de209e1e78" is_pre_election: true
I20260621 07:02:32.215885 20660 raft_consensus.cc:482] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:32.215986 20660 raft_consensus.cc:3037] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:32.215710 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.202789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.2.245.194:58585)
0621 07:02:32.203659 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.203677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:32.203809 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:32.206113 (+ 2304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:32.206119 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:32.206150 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:32.206516 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.206530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.208136 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.208140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.210825 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:32.210839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.212970 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.212977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:32.213160 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.214524 (+ 1364us) client_negotiation.cc:769] Sending connection context
0621 07:02:32.214984 (+ 460us) client_negotiation.cc:240] Negotiation successful
0621 07:02:32.215135 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":178,"thread_start_us":2125,"threads_started":1}
I20260621 07:02:32.215996 20589 raft_consensus.cc:2454] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b211485a348d465ca4f5c9a3fa2357bb in term 0.
I20260621 07:02:32.216470 20391 leader_election.cc:302] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c3c51f76c64ffb849e00de209e1e78, b211485a348d465ca4f5c9a3fa2357bb; no voters:
I20260621 07:02:32.216894 20659 raft_consensus.cc:2781] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:32.217051 20659 raft_consensus.cc:482] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:32.217160 20659 raft_consensus.cc:3037] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:32.217052 20322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "b211485a348d465ca4f5c9a3fa2357bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" is_pre_election: true
I20260621 07:02:32.218881 20660 raft_consensus.cc:504] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:32.219414 20322 raft_consensus.cc:2379] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b211485a348d465ca4f5c9a3fa2357bb in current term 1: Already voted for candidate be1b7f5b0af94c66a9387efaff3f62b3 in this term.
I20260621 07:02:32.219556 20659 raft_consensus.cc:504] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:32.220304 20456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b211485a348d465ca4f5c9a3fa2357bb"
I20260621 07:02:32.220279 20589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c3c51f76c64ffb849e00de209e1e78"
I20260621 07:02:32.220482 20589 raft_consensus.cc:3037] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:32.220559 20456 raft_consensus.cc:2379] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be1b7f5b0af94c66a9387efaff3f62b3 in current term 1: Already voted for candidate b211485a348d465ca4f5c9a3fa2357bb in this term.
I20260621 07:02:32.220769 20321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "b211485a348d465ca4f5c9a3fa2357bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be1b7f5b0af94c66a9387efaff3f62b3"
I20260621 07:02:32.221102 20321 raft_consensus.cc:2379] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b211485a348d465ca4f5c9a3fa2357bb in current term 1: Already voted for candidate be1b7f5b0af94c66a9387efaff3f62b3 in this term.
I20260621 07:02:32.221352 20660 leader_election.cc:288] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [CANDIDATE]: Term 1 election: Requested vote from peers b211485a348d465ca4f5c9a3fa2357bb (127.2.245.194:36321), 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353)
I20260621 07:02:32.221472 20659 leader_election.cc:288] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [CANDIDATE]: Term 1 election: Requested vote from peers 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353), be1b7f5b0af94c66a9387efaff3f62b3 (127.2.245.193:35643)
I20260621 07:02:32.221915 20588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6520b679a3694c678e76981ebc226452" candidate_uuid: "b211485a348d465ca4f5c9a3fa2357bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c3c51f76c64ffb849e00de209e1e78"
I20260621 07:02:32.222786 20391 leader_election.cc:302] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b211485a348d465ca4f5c9a3fa2357bb; no voters: 68c3c51f76c64ffb849e00de209e1e78, be1b7f5b0af94c66a9387efaff3f62b3
I20260621 07:02:32.223078 20659 raft_consensus.cc:2726] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260621 07:02:32.223610 20589 raft_consensus.cc:2454] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be1b7f5b0af94c66a9387efaff3f62b3 in term 1.
I20260621 07:02:32.223973 20257 leader_election.cc:302] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c3c51f76c64ffb849e00de209e1e78, be1b7f5b0af94c66a9387efaff3f62b3; no voters: b211485a348d465ca4f5c9a3fa2357bb
I20260621 07:02:32.224292 20660 raft_consensus.cc:2781] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:32.224800 20660 raft_consensus.cc:686] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 LEADER]: Becoming Leader. State: Replica: be1b7f5b0af94c66a9387efaff3f62b3, State: Running, Role: LEADER
I20260621 07:02:32.225332 20660 consensus_queue.cc:237] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } }
I20260621 07:02:32.230404 20178 catalog_manager.cc:5495] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to be1b7f5b0af94c66a9387efaff3f62b3 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be1b7f5b0af94c66a9387efaff3f62b3" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35643 } health_report { overall_health: HEALTHY } } }
I20260621 07:02:32.285164 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.279431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.2.245.193:35643)
0621 07:02:32.279686 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.279693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.279803 (+ 110us) server_negotiation.cc:407] Connection header received
0621 07:02:32.280076 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.280080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.280174 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.280332 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:32.280724 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.281478 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.282920 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.283381 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.283518 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.284047 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:32.284562 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:32.284732 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.284788 (+ 56us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.284869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260621 07:02:32.293697 20589 raft_consensus.cc:1261] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 1 FOLLOWER]: Refusing update from remote peer be1b7f5b0af94c66a9387efaff3f62b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:32.293713 20456 raft_consensus.cc:1261] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Refusing update from remote peer be1b7f5b0af94c66a9387efaff3f62b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:32.294633 20666 consensus_queue.cc:1035] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c3c51f76c64ffb849e00de209e1e78" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:32.295018 20660 consensus_queue.cc:1035] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b211485a348d465ca4f5c9a3fa2357bb" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 36321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:32.306435 20669 mvcc.cc:205] Tried to move back new op lower bound from 7299175842984644608 to 7299175842719752192. Current Snapshot: MvccSnapshot[applied={T|T < 7299175842984644608}]
I20260621 07:02:32.321225 20178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:32.321522 20178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:32.325234 20178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8138a71f22db4ade894dfa2d2b9febc1: Sending DeleteTablet for 3 replicas of tablet 6520b679a3694c678e76981ebc226452
I20260621 07:02:32.326634 20302 tablet_service.cc:1514] Processing DeleteTablet for tablet 6520b679a3694c678e76981ebc226452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:32 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:38030
I20260621 07:02:32.326634 20436 tablet_service.cc:1514] Processing DeleteTablet for tablet 6520b679a3694c678e76981ebc226452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:32 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:39320
I20260621 07:02:32.326915 20569 tablet_service.cc:1514] Processing DeleteTablet for tablet 6520b679a3694c678e76981ebc226452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:32 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:56242
I20260621 07:02:32.328874 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20239
I20260621 07:02:32.337777 20681 tablet_replica.cc:330] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: stopping tablet replica
I20260621 07:02:32.338387 20681 raft_consensus.cc:2229] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:32.339032 20681 raft_consensus.cc:2258] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:32.340082 20679 tablet_replica.cc:330] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: stopping tablet replica
I20260621 07:02:32.340582 20679 raft_consensus.cc:2229] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:32.341115 20679 raft_consensus.cc:2258] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:32.344278 20679 ts_tablet_manager.cc:1905] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:32.351361 20681 ts_tablet_manager.cc:1905] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 21 07:02:32 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:02:32.358029 20680 tablet_replica.cc:330] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: stopping tablet replica
I20260621 07:02:32.358537 20680 raft_consensus.cc:2229] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:32.358649 20679 ts_tablet_manager.cc:1918] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:32.358809 20679 log.cc:1196] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/6520b679a3694c678e76981ebc226452
I20260621 07:02:32.359246 20680 raft_consensus.cc:2258] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:32.359386 20679 ts_tablet_manager.cc:1939] T 6520b679a3694c678e76981ebc226452 P b211485a348d465ca4f5c9a3fa2357bb: Deleting consensus metadata
I20260621 07:02:32.361259 20163 catalog_manager.cc:4841] TS b211485a348d465ca4f5c9a3fa2357bb (127.2.245.194:36321): tablet 6520b679a3694c678e76981ebc226452 (table test-table [id=f5d008412b79450ca1817ae8e33ebb18]) successfully deleted
I20260621 07:02:32.361398 20680 ts_tablet_manager.cc:1905] T 6520b679a3694c678e76981ebc226452 P be1b7f5b0af94c66a9387efaff3f62b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:32.363242 20681 ts_tablet_manager.cc:1918] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:32.363422 20681 log.cc:1196] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/6520b679a3694c678e76981ebc226452
I20260621 07:02:32.363929 20681 ts_tablet_manager.cc:1939] T 6520b679a3694c678e76981ebc226452 P 68c3c51f76c64ffb849e00de209e1e78: Deleting consensus metadata
I20260621 07:02:32.364645 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.330547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.2.245.193:35643)
0621 07:02:32.330766 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.330775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.342616 (+ 11841us) server_negotiation.cc:407] Connection header received
0621 07:02:32.342962 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.342970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.343068 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.343161 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:32.345139 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.346297 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.348274 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.348982 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.349110 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.355212 (+ 6102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:32.355235 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:32.355257 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:32.355320 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:32.359650 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.360815 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.360824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.360827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.360980 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.361610 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.361615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.361618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.362150 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:32.362469 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.364041 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.364302 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:02:32.365585 20362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:32.365697 20164 catalog_manager.cc:4841] TS 68c3c51f76c64ffb849e00de209e1e78 (127.2.245.195:42353): tablet 6520b679a3694c678e76981ebc226452 (table test-table [id=f5d008412b79450ca1817ae8e33ebb18]) successfully deleted
I20260621 07:02:32.467896 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20239
I20260621 07:02:32.486168 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20373
W20260621 07:02:32.486133 20162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:35643 (error 108)
I20260621 07:02:32.488212 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.487449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38070)
0621 07:02:32.487920 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.488033 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":270,"threads_started":1}
W20260621 07:02:32.488459 20162 catalog_manager.cc:4584] TS be1b7f5b0af94c66a9387efaff3f62b3 (127.2.245.193:35643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6520b679a3694c678e76981ebc226452: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Jun 21 07:02:32 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:02:32.501847 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.487048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.2.245.194:36321)
0621 07:02:32.487340 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.487345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.487363 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:02:32.487517 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.487521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.487619 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.487745 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:32.489313 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.490010 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.491463 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.491854 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.491997 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.496110 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:32.496129 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:32.496148 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:32.496195 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:32.498847 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.499530 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.499543 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.499546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.499686 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.500142 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.500147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.500150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.500614 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:32.500913 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.501251 (+ 338us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.501482 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260621 07:02:32.502485 20496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:32.542416 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.541931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38078)
0621 07:02:32.542170 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.542266 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260621 07:02:32.543661 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.543230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38088)
0621 07:02:32.543407 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.543480 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260621 07:02:32.595964 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20373
I20260621 07:02:32.604137 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.603691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38098)
0621 07:02:32.603893 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.603985 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260621 07:02:32.605198 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.604839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38112)
0621 07:02:32.604990 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.605063 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 07:02:32.613586 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20506
Jun 21 07:02:32 dist-test-slave-9zdv krb5kdc[20133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:02:32.629087 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.614270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.2.245.195:42353)
0621 07:02:32.614467 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:32.614474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:32.614567 (+ 93us) server_negotiation.cc:407] Connection header received
0621 07:02:32.614762 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:32.614767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:32.614845 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:32.614973 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:32.616320 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.617000 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.618298 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:32.618721 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:32.618850 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:32.623347 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:32.623361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:32.623377 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:32.623409 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:32.626183 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.626896 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.626904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.626906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.627142 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:32.627540 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:32.627547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:32.627550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:32.627894 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:32.628190 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:32.628510 (+ 320us) server_negotiation.cc:299] Negotiation successful
0621 07:02:32.628769 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:02:32.629757 20629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:32.685884 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.685372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38122)
0621 07:02:32.685624 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.685724 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260621 07:02:32.687279 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:32.686670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35643 (local address 127.0.0.1:38136)
0621 07:02:32.686883 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:32.687003 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:35643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260621 07:02:32.730456 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20506
I20260621 07:02:32.747265 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20146
I20260621 07:02:32.748422 20208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:32.860116 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20146
2026-06-21T07:02:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4615 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-21T07:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:32Z Disabled control of system clock
I20260621 07:02:32.938264 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36317
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:38341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:36317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:33.268702 20692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:33.268980 20692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:33.269063 20692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:33.276283 20692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:33.276388 20692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:33.276427 20692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:33.276476 20692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:33.276521 20692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:33.285567 20692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:36317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:33.286792 20692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:33.288496 20692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:33.300693 20700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:33.300797 20697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:33.300740 20698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:33.304327 20692 server_base.cc:1034] running on GCE node
I20260621 07:02:33.305106 20692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:33.306453 20692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:33.307653 20692 hybrid_clock.cc:648] HybridClock initialized: now 1782025353307592 us; error 58 us; skew 500 ppm
I20260621 07:02:33.307956 20692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:33.313213 20692 webserver.cc:460] Webserver started at http://127.2.245.254:41199/ using document root <none> and password file <none>
I20260621 07:02:33.313891 20692 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:33.313979 20692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:33.314260 20692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:33.317154 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "dca2f72d70ad4fa18562d8d02726a13e"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "34c7746f872509e74aa3c233eb72d438"
server_key_iv: "14ba204667e315f93302110cecd57288"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.317888 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "dca2f72d70ad4fa18562d8d02726a13e"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "34c7746f872509e74aa3c233eb72d438"
server_key_iv: "14ba204667e315f93302110cecd57288"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.323041 20692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260621 07:02:33.326449 20706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:33.327880 20692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:33.328083 20692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "dca2f72d70ad4fa18562d8d02726a13e"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "34c7746f872509e74aa3c233eb72d438"
server_key_iv: "14ba204667e315f93302110cecd57288"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.328318 20692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:33.345247 20692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:33.346086 20692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:33.346383 20692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:33.365694 20692 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:36317
I20260621 07:02:33.365701 20757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:36317 every 8 connection(s)
I20260621 07:02:33.367097 20692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:33.370752 20758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:33.372141 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20692
I20260621 07:02:33.372335 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:33.372929 3031 external_mini_cluster.cc:1428] Setting key 1eed5e45ad0f23cd6089e819c158fe12
I20260621 07:02:33.383991 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: Bootstrap starting.
I20260621 07:02:33.386457 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:33.374871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.2.245.254:36317)
0621 07:02:33.375952 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:33.375966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:33.376006 (+ 40us) server_negotiation.cc:407] Connection header received
0621 07:02:33.377717 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:33.377738 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:33.378192 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:33.378592 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:33.379255 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.380571 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.382040 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.382719 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.382919 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:33.384201 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:33.384217 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:33.384231 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:33.384770 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:33.384846 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:33.384867 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:33.385004 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:33.385227 (+ 223us) server_negotiation.cc:299] Negotiation successful
0621 07:02:33.385545 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":407,"threads_started":1}
I20260621 07:02:33.387599 20758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:33.388667 20758 log.cc:824] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:33.391577 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: No bootstrap required, opened a new log
I20260621 07:02:33.398772 20758 raft_consensus.cc:348] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } }
I20260621 07:02:33.399191 20758 raft_consensus.cc:374] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:33.399349 20758 raft_consensus.cc:729] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca2f72d70ad4fa18562d8d02726a13e, State: Initialized, Role: FOLLOWER
I20260621 07:02:33.400007 20758 consensus_queue.cc:260] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } }
I20260621 07:02:33.400264 20758 raft_consensus.cc:388] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:33.400396 20758 raft_consensus.cc:482] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:33.400568 20758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:33.402724 20758 raft_consensus.cc:504] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } }
I20260621 07:02:33.403249 20758 leader_election.cc:302] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dca2f72d70ad4fa18562d8d02726a13e; no voters:
I20260621 07:02:33.403878 20758 leader_election.cc:288] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:33.404007 20763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:33.404824 20763 raft_consensus.cc:686] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [term 1 LEADER]: Becoming Leader. State: Replica: dca2f72d70ad4fa18562d8d02726a13e, State: Running, Role: LEADER
I20260621 07:02:33.405382 20763 consensus_queue.cc:237] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } }
I20260621 07:02:33.406015 20758 sys_catalog.cc:564] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:33.408318 20765 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dca2f72d70ad4fa18562d8d02726a13e. Latest consensus state: current_term: 1 leader_uuid: "dca2f72d70ad4fa18562d8d02726a13e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } } }
I20260621 07:02:33.408718 20765 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:33.408447 20764 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dca2f72d70ad4fa18562d8d02726a13e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca2f72d70ad4fa18562d8d02726a13e" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36317 } } }
I20260621 07:02:33.408850 20764 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:33.411288 20769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:33.416154 20769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:33.425000 20769 catalog_manager.cc:1347] Generated new cluster ID: 6ce68de9d37643b3b46f4f22ebf38dc7
I20260621 07:02:33.425133 20769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:33.468559 20769 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:33.470165 20769 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:33.478363 20769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: Generated new TSK 0
I20260621 07:02:33.479326 20769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:33.484273 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--builtin_ntp_servers=127.2.245.212:38341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:33.821234 20782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:33.821496 20782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:33.821713 20782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:33.829205 20782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:33.829312 20782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:33.829487 20782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:33.839166 20782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:33.840456 20782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:33.842233 20782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:33.855697 20788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:33.855727 20787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:33.855723 20790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:33.856994 20782 server_base.cc:1034] running on GCE node
I20260621 07:02:33.857602 20782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:33.858541 20782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:33.859771 20782 hybrid_clock.cc:648] HybridClock initialized: now 1782025353859721 us; error 45 us; skew 500 ppm
I20260621 07:02:33.860103 20782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:33.862519 20782 webserver.cc:460] Webserver started at http://127.2.245.193:40359/ using document root <none> and password file <none>
I20260621 07:02:33.863318 20782 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:33.863463 20782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:33.863813 20782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:33.866647 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "7a53655c37934a7883a0a340eab4c828"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "42d5f596aa9068101ad1244d85cf2ff5"
server_key_iv: "dbcf7f76485f038cc7907cc5b98cdb01"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.867507 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "7a53655c37934a7883a0a340eab4c828"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "42d5f596aa9068101ad1244d85cf2ff5"
server_key_iv: "dbcf7f76485f038cc7907cc5b98cdb01"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.872433 20782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:02:33.875934 20796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:33.877350 20782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:33.877475 20782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "7a53655c37934a7883a0a340eab4c828"
format_stamp: "Formatted at 2026-06-21 07:02:33 on dist-test-slave-9zdv"
server_key: "42d5f596aa9068101ad1244d85cf2ff5"
server_key_iv: "dbcf7f76485f038cc7907cc5b98cdb01"
server_key_version: "encryptionkey@0"
I20260621 07:02:33.877667 20782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:33.920967 20782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:33.921840 20782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:33.922091 20782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:33.923110 20782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:33.924599 20782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:33.924695 20782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:33.924772 20782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:33.924826 20782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:33.958207 20782 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:39163
I20260621 07:02:33.958261 20908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:39163 every 8 connection(s)
I20260621 07:02:33.959659 20782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:33.969055 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20782
I20260621 07:02:33.969254 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:33.969780 3031 external_mini_cluster.cc:1428] Setting key 68ffdfbc80ba423a30fb0e67afe505df
I20260621 07:02:33.972286 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:33.962333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:39681 (local address 127.2.245.254:36317)
0621 07:02:33.962806 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:33.962813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:33.964054 (+ 1241us) server_negotiation.cc:407] Connection header received
0621 07:02:33.965778 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:33.965783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:33.965862 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:33.965960 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:33.967032 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.967741 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.969256 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.969853 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.969982 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:33.971428 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:33.971433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:33.971436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:33.971452 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:33.971490 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:33.971502 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:33.971595 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:33.971830 (+ 235us) server_negotiation.cc:299] Negotiation successful
0621 07:02:33.971923 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260621 07:02:33.972887 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--builtin_ntp_servers=127.2.245.212:38341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:33.973361 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:33.963003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36317 (local address 127.2.245.193:39681)
0621 07:02:33.963897 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:33.963939 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:33.965501 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:33.966181 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:33.966189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:33.966274 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:33.966821 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:33.966837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.967874 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.967877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:33.969062 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:33.969073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:33.970050 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:33.970055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:33.970262 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:33.971100 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:33.971103 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:33.971155 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:33.971250 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:33.971606 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:33.971618 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:33.971626 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:02:33.971820 (+ 194us) client_negotiation.cc:240] Negotiation successful
0621 07:02:33.972197 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260621 07:02:33.974922 20909 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36317
I20260621 07:02:33.975253 20909 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:33.976145 20909 heartbeater.cc:511] Master 127.2.245.254:36317 requested a full tablet report, sending...
I20260621 07:02:33.978382 20723 ts_manager.cc:194] Registered new tserver with Master: 7a53655c37934a7883a0a340eab4c828 (127.2.245.193:39163)
I20260621 07:02:33.979907 20723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.193:39681
W20260621 07:02:34.296027 20913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:34.296356 20913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:34.296519 20913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:34.303894 20913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:34.304061 20913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:34.304240 20913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:34.313256 20913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:34.314568 20913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:34.316421 20913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:34.329447 20921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:34.329493 20919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:34.329496 20918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:34.334153 20913 server_base.cc:1034] running on GCE node
I20260621 07:02:34.334806 20913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:34.335855 20913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:34.337062 20913 hybrid_clock.cc:648] HybridClock initialized: now 1782025354337009 us; error 47 us; skew 500 ppm
I20260621 07:02:34.337407 20913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:34.339730 20913 webserver.cc:460] Webserver started at http://127.2.245.194:35363/ using document root <none> and password file <none>
I20260621 07:02:34.340435 20913 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:34.340546 20913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:34.340873 20913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:34.343824 20913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "5687d06dba7f6bf9ea93e02227d0857d"
server_key_iv: "ef4e55baceee4b10ebea6abff403b493"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.344589 20913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "5687d06dba7f6bf9ea93e02227d0857d"
server_key_iv: "ef4e55baceee4b10ebea6abff403b493"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.349678 20913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:02:34.353068 20927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.354346 20913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:34.354513 20913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "5687d06dba7f6bf9ea93e02227d0857d"
server_key_iv: "ef4e55baceee4b10ebea6abff403b493"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.354737 20913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:34.370414 20913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:34.371310 20913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:34.371603 20913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:34.372615 20913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:34.374092 20913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:34.374229 20913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.374349 20913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:34.374433 20913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.408804 20913 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:45445
I20260621 07:02:34.408890 21039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:45445 every 8 connection(s)
I20260621 07:02:34.410113 20913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:34.417796 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 20913
I20260621 07:02:34.418001 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:34.418560 3031 external_mini_cluster.cc:1428] Setting key 7cadfa47905541d3c0b9ca080dfaaf57
I20260621 07:02:34.421717 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--builtin_ntp_servers=127.2.245.212:38341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:34.424383 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.412842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:42595 (local address 127.2.245.254:36317)
0621 07:02:34.413030 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.413035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.414426 (+ 1391us) server_negotiation.cc:407] Connection header received
0621 07:02:34.416217 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.416221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.416325 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.416417 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:34.417730 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.418841 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.421094 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.421778 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.421928 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.423429 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:34.423438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:34.423445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:34.423470 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:34.423511 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:34.423530 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:34.423625 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.423873 (+ 248us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.423987 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:02:34.425237 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.413398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36317 (local address 127.2.245.194:42595)
0621 07:02:34.414268 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:34.414303 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:34.415968 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:34.416698 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:34.416707 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:34.416794 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:34.417524 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.417546 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.419039 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.419043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.420944 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.420957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.422093 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.422109 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.422306 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.423085 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:34.423087 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:34.423139 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:34.423219 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:34.423632 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:34.423640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:34.423647 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 07:02:34.423795 (+ 148us) client_negotiation.cc:240] Negotiation successful
0621 07:02:34.424115 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":275,"threads_started":1}
I20260621 07:02:34.426800 21040 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36317
I20260621 07:02:34.427151 21040 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:34.428066 21040 heartbeater.cc:511] Master 127.2.245.254:36317 requested a full tablet report, sending...
I20260621 07:02:34.429709 20723 ts_manager.cc:194] Registered new tserver with Master: 9b5e7404ca814af3a7e8ac84d1bc1cf7 (127.2.245.194:45445)
I20260621 07:02:34.430742 20723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.194:42595
W20260621 07:02:34.746724 21043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:34.747064 21043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:34.747239 21043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:34.754091 21043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:34.754226 21043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:34.754393 21043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:34.764822 21043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:36317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:34.766132 21043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:34.767941 21043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:34.780897 21048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:34.780977 21049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:34.781343 21051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:34.782212 21043 server_base.cc:1034] running on GCE node
I20260621 07:02:34.782768 21043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:34.783689 21043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:34.784888 21043 hybrid_clock.cc:648] HybridClock initialized: now 1782025354784857 us; error 26 us; skew 500 ppm
I20260621 07:02:34.785303 21043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:34.787520 21043 webserver.cc:460] Webserver started at http://127.2.245.195:41559/ using document root <none> and password file <none>
I20260621 07:02:34.788195 21043 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:34.788311 21043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:34.788640 21043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:34.791472 21043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "50b4414a81984b2788be3756fc58be82"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "d571045967298f276de74e38f5a8ce51"
server_key_iv: "5cda25066b4ff0de3655f5549699f710"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.792274 21043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "50b4414a81984b2788be3756fc58be82"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "d571045967298f276de74e38f5a8ce51"
server_key_iv: "5cda25066b4ff0de3655f5549699f710"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.797165 21043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260621 07:02:34.800535 21057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.801821 21043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 07:02:34.801987 21043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "50b4414a81984b2788be3756fc58be82"
format_stamp: "Formatted at 2026-06-21 07:02:34 on dist-test-slave-9zdv"
server_key: "d571045967298f276de74e38f5a8ce51"
server_key_iv: "5cda25066b4ff0de3655f5549699f710"
server_key_version: "encryptionkey@0"
I20260621 07:02:34.802213 21043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:34.822422 21043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:34.823226 21043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:34.823523 21043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:34.824411 21043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:34.825784 21043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:34.825867 21043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.826028 21043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:34.826125 21043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:34.858757 21043 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:45903
I20260621 07:02:34.858865 21169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:45903 every 8 connection(s)
I20260621 07:02:34.860201 21043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:34.867579 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21043
I20260621 07:02:34.867754 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:34.868211 3031 external_mini_cluster.cc:1428] Setting key ff5b2e734d03a50d47cd6412df82e47b
I20260621 07:02:34.873092 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.862663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:33775 (local address 127.2.245.254:36317)
0621 07:02:34.862834 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.862838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.864328 (+ 1490us) server_negotiation.cc:407] Connection header received
0621 07:02:34.866128 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.866135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.866236 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.866349 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:34.867484 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.868428 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.870326 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.870913 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.871071 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.872453 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:34.872457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:34.872461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:34.872478 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:34.872508 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:34.872520 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:34.872589 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.872792 (+ 203us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.872866 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:02:34.874037 21172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.863274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:36317 (local address 127.2.245.195:33775)
0621 07:02:34.864173 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:34.864209 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:34.865865 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:34.866577 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:34.866585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:34.866670 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:34.867322 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.867338 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.868598 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.868602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.870169 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.870181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.871153 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.871160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.871385 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.872125 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:34.872128 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:34.872192 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:34.872286 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:34.872605 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:34.872612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:34.872617 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 07:02:34.872766 (+ 149us) client_negotiation.cc:240] Negotiation successful
0621 07:02:34.873036 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":262,"threads_started":1}
I20260621 07:02:34.875305 21170 heartbeater.cc:347] Connected to a master server at 127.2.245.254:36317
I20260621 07:02:34.875546 21170 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:34.876227 21170 heartbeater.cc:511] Master 127.2.245.254:36317 requested a full tablet report, sending...
I20260621 07:02:34.877482 20723 ts_manager.cc:194] Registered new tserver with Master: 50b4414a81984b2788be3756fc58be82 (127.2.245.195:45903)
I20260621 07:02:34.878305 20723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.195:33775
I20260621 07:02:34.883212 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:34.892585 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.887072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.2.245.254:36317)
0621 07:02:34.887232 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.887237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.887686 (+ 449us) server_negotiation.cc:407] Connection header received
0621 07:02:34.887830 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.887833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.887888 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.887966 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:34.888416 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.889088 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.890458 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.890822 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.890929 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.892051 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:34.892055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:34.892058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:34.892075 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:34.892104 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:34.892115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:34.892183 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.892307 (+ 124us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.892375 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:02:34.896103 20723 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:34.903103 20723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:34.942649 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.915685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.2.245.193:39163)
0621 07:02:34.934520 (+ 18835us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.934530 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.934560 (+ 30us) server_negotiation.cc:407] Connection header received
0621 07:02:34.934662 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.934671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.934921 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.935422 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:34.935698 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937349 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.940496 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.941423 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.941581 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.941845 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.942141 (+ 296us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.942307 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18684,"thread_start_us":18595,"threads_started":1}
I20260621 07:02:34.942704 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.915408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39800)
0621 07:02:34.930357 (+ 14949us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:34.930417 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:34.930641 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:34.935148 (+ 4507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:34.935164 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:34.935193 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:34.935542 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.935556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937460 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.937467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.940255 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.940270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.941669 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.941676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.941878 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.941933 (+ 55us) client_negotiation.cc:769] Sending connection context
0621 07:02:34.942080 (+ 147us) client_negotiation.cc:240] Negotiation successful
0621 07:02:34.942267 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14634,"mutex_wait_us":329,"thread_start_us":15441,"threads_started":1}
I20260621 07:02:34.943266 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.915897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:45445 (local address 127.0.0.1:49712)
0621 07:02:34.933437 (+ 17540us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:34.933451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:34.933540 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:34.935499 (+ 1959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:34.935504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:34.935535 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:34.936121 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.936135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937381 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.937389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.940452 (+ 3063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.940464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.942210 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.942216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.942597 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.942637 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 07:02:34.942765 (+ 128us) client_negotiation.cc:240] Negotiation successful
0621 07:02:34.942898 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17394,"mutex_wait_us":186,"thread_start_us":17763,"threads_started":1}
I20260621 07:02:34.944052 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.916229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:45903 (local address 127.0.0.1:35298)
0621 07:02:34.933878 (+ 17649us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:34.933890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:34.934020 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:34.935047 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:34.935057 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:34.935095 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:34.936038 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.936054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937845 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.937855 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.939914 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:34.939926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.943128 (+ 3202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.943138 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:34.943482 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.943513 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 07:02:34.943601 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 07:02:34.943729 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17564,"mutex_wait_us":85,"thread_start_us":16940,"threads_started":1}
I20260621 07:02:34.944154 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.916627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.2.245.194:45445)
0621 07:02:34.934880 (+ 18253us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.934900 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.934927 (+ 27us) server_negotiation.cc:407] Connection header received
0621 07:02:34.935049 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.935055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.935279 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.935893 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:34.936293 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937233 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.940649 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.942012 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.942179 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.942383 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.943633 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.943831 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18090,"thread_start_us":18978,"threads_started":1}
I20260621 07:02:34.944850 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:34.916838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.2.245.195:45903)
0621 07:02:34.932977 (+ 16139us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:34.932990 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:34.934001 (+ 1011us) server_negotiation.cc:407] Connection header received
0621 07:02:34.934197 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:34.934209 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:34.934509 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:34.934716 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:34.936285 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.937703 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.940116 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:34.941412 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:34.941550 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:34.941871 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:34.944417 (+ 2546us) server_negotiation.cc:299] Negotiation successful
0621 07:02:34.944541 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15938,"thread_start_us":15801,"threads_started":1}
I20260621 07:02:34.946972 20844 tablet_service.cc:1467] Processing CreateTablet for tablet ac2de808a75f42fb8b681dba26f1c76a (DEFAULT_TABLE table=test-table [id=ab75b3af67a5470bb308457a1ff22fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:34.947407 20975 tablet_service.cc:1467] Processing CreateTablet for tablet ac2de808a75f42fb8b681dba26f1c76a (DEFAULT_TABLE table=test-table [id=ab75b3af67a5470bb308457a1ff22fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:34.948753 20975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2de808a75f42fb8b681dba26f1c76a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:34.949030 20844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2de808a75f42fb8b681dba26f1c76a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:34.948684 21105 tablet_service.cc:1467] Processing CreateTablet for tablet ac2de808a75f42fb8b681dba26f1c76a (DEFAULT_TABLE table=test-table [id=ab75b3af67a5470bb308457a1ff22fb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:34.949968 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac2de808a75f42fb8b681dba26f1c76a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:34.965890 21188 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Bootstrap starting.
I20260621 07:02:34.967334 21190 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: Bootstrap starting.
I20260621 07:02:34.971330 21188 tablet_bootstrap.cc:654] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:34.972694 21188 log.cc:824] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:34.972978 21189 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Bootstrap starting.
I20260621 07:02:34.976015 21190 tablet_bootstrap.cc:654] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:34.976523 21189 tablet_bootstrap.cc:654] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:34.977011 21188 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: No bootstrap required, opened a new log
I20260621 07:02:34.977245 21190 log.cc:824] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:34.977303 21188 ts_tablet_manager.cc:1397] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260621 07:02:34.977710 21189 log.cc:824] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:34.980033 21190 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: No bootstrap required, opened a new log
I20260621 07:02:34.980286 21190 ts_tablet_manager.cc:1397] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260621 07:02:34.980824 21189 tablet_bootstrap.cc:492] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: No bootstrap required, opened a new log
I20260621 07:02:34.981089 21189 ts_tablet_manager.cc:1397] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260621 07:02:34.982999 20909 heartbeater.cc:503] Master 127.2.245.254:36317 was elected leader, sending a full tablet report...
I20260621 07:02:34.987952 21188 raft_consensus.cc:348] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.988408 21188 raft_consensus.cc:374] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:34.988507 21188 raft_consensus.cc:729] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5e7404ca814af3a7e8ac84d1bc1cf7, State: Initialized, Role: FOLLOWER
I20260621 07:02:34.989108 21188 consensus_queue.cc:260] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.989779 21190 raft_consensus.cc:348] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.990185 21190 raft_consensus.cc:374] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:34.990213 21040 heartbeater.cc:503] Master 127.2.245.254:36317 was elected leader, sending a full tablet report...
I20260621 07:02:34.990283 21190 raft_consensus.cc:729] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a53655c37934a7883a0a340eab4c828, State: Initialized, Role: FOLLOWER
I20260621 07:02:34.990876 21188 ts_tablet_manager.cc:1428] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260621 07:02:34.990967 21190 consensus_queue.cc:260] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.991437 21189 raft_consensus.cc:348] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.991871 21189 raft_consensus.cc:374] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:34.991941 21189 raft_consensus.cc:729] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b4414a81984b2788be3756fc58be82, State: Initialized, Role: FOLLOWER
I20260621 07:02:34.992421 21189 consensus_queue.cc:260] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:34.993597 21170 heartbeater.cc:503] Master 127.2.245.254:36317 was elected leader, sending a full tablet report...
I20260621 07:02:34.994065 21189 ts_tablet_manager.cc:1428] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260621 07:02:34.996236 21190 ts_tablet_manager.cc:1428] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260621 07:02:34.999028 21195 raft_consensus.cc:482] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:34.999334 21195 raft_consensus.cc:504] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:35.000792 21195 leader_election.cc:288] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b5e7404ca814af3a7e8ac84d1bc1cf7 (127.2.245.194:45445), 50b4414a81984b2788be3756fc58be82 (127.2.245.195:45903)
I20260621 07:02:35.007930 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.001555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:48593 (local address 127.2.245.195:45903)
0621 07:02:35.001754 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.001760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.001779 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:02:35.001976 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.001983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.002069 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.002212 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:35.002772 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.003801 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.005841 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.006932 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.007134 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.007212 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.007581 (+ 369us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.007666 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:02:35.008046 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.001020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:45903 (local address 127.2.245.193:48593)
0621 07:02:35.001681 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.001702 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:35.001826 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:35.002212 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:35.002217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:35.002248 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:35.002623 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:35.002638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.003922 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.003928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:35.005599 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:35.005611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.007210 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.007219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:35.007398 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.007446 (+ 48us) client_negotiation.cc:769] Sending connection context
0621 07:02:35.007542 (+ 96us) client_negotiation.cc:240] Negotiation successful
0621 07:02:35.007667 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":342,"threads_started":1}
I20260621 07:02:35.008675 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.001660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:43197 (local address 127.2.245.194:45445)
0621 07:02:35.001874 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.001883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.002531 (+ 648us) server_negotiation.cc:407] Connection header received
0621 07:02:35.002679 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.002687 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.002777 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.002854 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:35.003503 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.004420 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.006452 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.007662 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.007809 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.007889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.008293 (+ 404us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.008431 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:02:35.009151 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.001133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:45445 (local address 127.2.245.193:43197)
0621 07:02:35.002434 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.002448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:35.002549 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:35.002898 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:35.002902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:35.002928 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:35.003319 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:35.003330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.004567 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.004571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:35.006284 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:35.006298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.007862 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.007868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:35.008102 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.008133 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 07:02:35.008846 (+ 713us) client_negotiation.cc:240] Negotiation successful
0621 07:02:35.008945 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":930,"threads_started":1}
I20260621 07:02:35.009063 21125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2de808a75f42fb8b681dba26f1c76a" candidate_uuid: "7a53655c37934a7883a0a340eab4c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b4414a81984b2788be3756fc58be82" is_pre_election: true
I20260621 07:02:35.009651 21125 raft_consensus.cc:2454] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a53655c37934a7883a0a340eab4c828 in term 0.
I20260621 07:02:35.010170 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2de808a75f42fb8b681dba26f1c76a" candidate_uuid: "7a53655c37934a7883a0a340eab4c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" is_pre_election: true
I20260621 07:02:35.010350 20798 leader_election.cc:302] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50b4414a81984b2788be3756fc58be82, 7a53655c37934a7883a0a340eab4c828; no voters:
I20260621 07:02:35.010707 20995 raft_consensus.cc:2454] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a53655c37934a7883a0a340eab4c828 in term 0.
I20260621 07:02:35.010772 21195 raft_consensus.cc:2781] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:35.010926 21195 raft_consensus.cc:482] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:35.011060 21195 raft_consensus.cc:3037] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:35.013448 21195 raft_consensus.cc:504] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:35.014117 21195 leader_election.cc:288] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [CANDIDATE]: Term 1 election: Requested vote from peers 9b5e7404ca814af3a7e8ac84d1bc1cf7 (127.2.245.194:45445), 50b4414a81984b2788be3756fc58be82 (127.2.245.195:45903)
I20260621 07:02:35.014524 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2de808a75f42fb8b681dba26f1c76a" candidate_uuid: "7a53655c37934a7883a0a340eab4c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7"
I20260621 07:02:35.014613 21125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac2de808a75f42fb8b681dba26f1c76a" candidate_uuid: "7a53655c37934a7883a0a340eab4c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b4414a81984b2788be3756fc58be82"
I20260621 07:02:35.014724 20995 raft_consensus.cc:3037] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:35.014853 21125 raft_consensus.cc:3037] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:35.017038 21125 raft_consensus.cc:2454] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a53655c37934a7883a0a340eab4c828 in term 1.
I20260621 07:02:35.017052 20995 raft_consensus.cc:2454] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a53655c37934a7883a0a340eab4c828 in term 1.
I20260621 07:02:35.017391 20798 leader_election.cc:302] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50b4414a81984b2788be3756fc58be82, 7a53655c37934a7883a0a340eab4c828; no voters:
I20260621 07:02:35.017675 21195 raft_consensus.cc:2781] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:35.018059 21195 raft_consensus.cc:686] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 1 LEADER]: Becoming Leader. State: Replica: 7a53655c37934a7883a0a340eab4c828, State: Running, Role: LEADER
I20260621 07:02:35.018518 21195 consensus_queue.cc:237] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } }
I20260621 07:02:35.022773 20722 catalog_manager.cc:5495] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a53655c37934a7883a0a340eab4c828 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "7a53655c37934a7883a0a340eab4c828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a53655c37934a7883a0a340eab4c828" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 39163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 } health_report { overall_health: UNKNOWN } } }
I20260621 07:02:35.044304 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.039224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.2.245.193:39163)
0621 07:02:35.039439 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.039447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.039557 (+ 110us) server_negotiation.cc:407] Connection header received
0621 07:02:35.039728 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.039732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.039820 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.039949 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:35.040259 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.040925 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.042276 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.042677 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.042766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.043292 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:35.043752 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:35.043860 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.043924 (+ 64us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.043985 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:02:35.052241 20995 raft_consensus.cc:1261] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 1 FOLLOWER]: Refusing update from remote peer 7a53655c37934a7883a0a340eab4c828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:35.052263 21125 raft_consensus.cc:1261] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 1 FOLLOWER]: Refusing update from remote peer 7a53655c37934a7883a0a340eab4c828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:35.053151 21199 consensus_queue.cc:1035] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5e7404ca814af3a7e8ac84d1bc1cf7" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 45445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:35.053516 21195 consensus_queue.cc:1035] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50b4414a81984b2788be3756fc58be82" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:35.064767 21202 mvcc.cc:205] Tried to move back new op lower bound from 7299175854283825152 to 7299175854159663104. Current Snapshot: MvccSnapshot[applied={T|T < 7299175854283825152}]
I20260621 07:02:35.078809 20722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:35.079133 20722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:35.083709 20722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dca2f72d70ad4fa18562d8d02726a13e: Sending DeleteTablet for 3 replicas of tablet ac2de808a75f42fb8b681dba26f1c76a
I20260621 07:02:35.084791 20844 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2de808a75f42fb8b681dba26f1c76a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:35 UTC) from {username='slave'} at 127.0.0.1:39800
I20260621 07:02:35.085148 20975 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2de808a75f42fb8b681dba26f1c76a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:35 UTC) from {username='slave'} at 127.0.0.1:49712
I20260621 07:02:35.085438 21105 tablet_service.cc:1514] Processing DeleteTablet for tablet ac2de808a75f42fb8b681dba26f1c76a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:35 UTC) from {username='slave'} at 127.0.0.1:35298
I20260621 07:02:35.086009 21213 tablet_replica.cc:330] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: stopping tablet replica
I20260621 07:02:35.086411 21213 raft_consensus.cc:2229] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:35.086807 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20782
I20260621 07:02:35.086894 21213 raft_consensus.cc:2258] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:35.088961 21213 ts_tablet_manager.cc:1905] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:35.097853 21213 ts_tablet_manager.cc:1918] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:35.098048 21213 log.cc:1196] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/ac2de808a75f42fb8b681dba26f1c76a
I20260621 07:02:35.098628 21213 ts_tablet_manager.cc:1939] T ac2de808a75f42fb8b681dba26f1c76a P 9b5e7404ca814af3a7e8ac84d1bc1cf7: Deleting consensus metadata
I20260621 07:02:35.101200 20710 catalog_manager.cc:4841] TS 9b5e7404ca814af3a7e8ac84d1bc1cf7 (127.2.245.194:45445): tablet ac2de808a75f42fb8b681dba26f1c76a (table test-table [id=ab75b3af67a5470bb308457a1ff22fb4]) successfully deleted
I20260621 07:02:35.103368 21214 tablet_replica.cc:330] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: stopping tablet replica
I20260621 07:02:35.103799 21214 raft_consensus.cc:2229] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:35.104169 21212 tablet_replica.cc:330] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828: stopping tablet replica
I20260621 07:02:35.104575 21212 raft_consensus.cc:2229] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:35.105245 21212 raft_consensus.cc:2258] T ac2de808a75f42fb8b681dba26f1c76a P 7a53655c37934a7883a0a340eab4c828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:35.105975 21214 raft_consensus.cc:2258] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:35.106761 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.089445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.2.245.193:39163)
0621 07:02:35.089629 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.089637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.098615 (+ 8978us) server_negotiation.cc:407] Connection header received
0621 07:02:35.099096 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.099119 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.099223 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.099299 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:35.099968 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.100862 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.102581 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.103322 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.103459 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.104931 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:35.104941 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:35.104962 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:35.105771 (+ 809us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:35.105830 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:35.105861 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:35.105945 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.106256 (+ 311us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.106364 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:02:35.108052 20904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:35.108289 21214 ts_tablet_manager.cc:1905] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:35.114873 21214 ts_tablet_manager.cc:1918] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:35.115068 21214 log.cc:1196] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/ac2de808a75f42fb8b681dba26f1c76a
I20260621 07:02:35.115516 21214 ts_tablet_manager.cc:1939] T ac2de808a75f42fb8b681dba26f1c76a P 50b4414a81984b2788be3756fc58be82: Deleting consensus metadata
I20260621 07:02:35.116935 20708 catalog_manager.cc:4841] TS 50b4414a81984b2788be3756fc58be82 (127.2.245.195:45903): tablet ac2de808a75f42fb8b681dba26f1c76a (table test-table [id=ab75b3af67a5470bb308457a1ff22fb4]) successfully deleted
I20260621 07:02:35.202652 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20782
W20260621 07:02:35.221323 20707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:39163 (error 108)
I20260621 07:02:35.222126 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20913
I20260621 07:02:35.223445 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.222904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39830)
0621 07:02:35.223106 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.223304 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260621 07:02:35.223708 20707 catalog_manager.cc:4584] TS 7a53655c37934a7883a0a340eab4c828 (127.2.245.193:39163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac2de808a75f42fb8b681dba26f1c76a: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
I20260621 07:02:35.229292 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.223093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.2.245.194:45445)
0621 07:02:35.223346 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.223352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.223371 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:02:35.223505 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.223510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.223656 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.223781 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:35.224197 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.224956 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.226302 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.226707 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.226843 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.227925 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:35.227934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:35.227948 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:35.228528 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:35.228578 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:35.228605 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:35.228732 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.228873 (+ 141us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.228979 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:02:35.230057 21035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:35.277796 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.277306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39836)
0621 07:02:35.277524 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.277627 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260621 07:02:35.278858 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.278510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39844)
0621 07:02:35.278667 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.278740 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260621 07:02:35.317811 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20913
I20260621 07:02:35.335292 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21043
I20260621 07:02:35.339406 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.338736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39850)
0621 07:02:35.339066 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.339189 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260621 07:02:35.340425 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.340114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39860)
0621 07:02:35.340251 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.340314 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 07:02:35.346365 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.340098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.2.245.195:45903)
0621 07:02:35.340336 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:35.340342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:35.340363 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:02:35.340474 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:35.340480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:35.340612 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:35.340745 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:35.341145 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.341891 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.343298 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:35.343711 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:35.343869 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:35.345002 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:35.345018 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:35.345033 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:35.345614 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:35.345672 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:35.345691 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:35.345815 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:35.345948 (+ 133us) server_negotiation.cc:299] Negotiation successful
0621 07:02:35.346058 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260621 07:02:35.347015 21165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:35.420957 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.420456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39876)
0621 07:02:35.420681 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.420783 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260621 07:02:35.422102 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:35.421693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:39163 (local address 127.0.0.1:39890)
0621 07:02:35.421921 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:35.421991 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260621 07:02:35.437167 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21043
I20260621 07:02:35.453821 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 20692
I20260621 07:02:35.454927 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:35.565045 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 20692
2026-06-21T07:02:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2701 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:35 dist-test-slave-9zdv krb5kdc[21223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:35 dist-test-slave-9zdv krb5kdc[21223](info): set up 2 sockets
Jun 21 07:02:35 dist-test-slave-9zdv krb5kdc[21223](info): commencing operation
krb5kdc: starting...
W20260621 07:02:36.671527 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:36 dist-test-slave-9zdv krb5kdc[21223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:02:36.924352 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46835
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:37.253815 21236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:37.254179 21236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:37.254336 21236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:37.261831 21236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:37.261989 21236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:37.262105 21236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:37.262202 21236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:37.262287 21236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:37.272426 21236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:46835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:37.273783 21236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:37.275141 21236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:37.286924 21244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:37.286870 21242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:37.286863 21241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:37.288002 21236 server_base.cc:1034] running on GCE node
I20260621 07:02:37.288699 21236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:37.290077 21236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:37.291277 21236 hybrid_clock.cc:648] HybridClock initialized: now 1782025357291229 us; error 47 us; skew 500 ppm
I20260621 07:02:37.291618 21236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:37 dist-test-slave-9zdv krb5kdc[21223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025357, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:37.294631 21236 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:02:37.296278 21236 webserver.cc:460] Webserver started at http://127.2.245.254:32849/ using document root <none> and password file <none>
I20260621 07:02:37.296912 21236 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:37.297012 21236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:37.297261 21236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:37.300213 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "25e81dcb731643019701e47b198813a1"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "6e113db4d784913712116487dd1af45a"
server_key_iv: "88fbca166642a912b36b678f9993afab"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.300901 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "25e81dcb731643019701e47b198813a1"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "6e113db4d784913712116487dd1af45a"
server_key_iv: "88fbca166642a912b36b678f9993afab"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.306066 21236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 07:02:37.309557 21251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:37.310914 21236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:02:37.311164 21236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "25e81dcb731643019701e47b198813a1"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "6e113db4d784913712116487dd1af45a"
server_key_iv: "88fbca166642a912b36b678f9993afab"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.311384 21236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:37.326570 21236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:37.330457 21236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:37.330739 21236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:37.349923 21236 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:46835
I20260621 07:02:37.349931 21302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:46835 every 8 connection(s)
I20260621 07:02:37.351377 21236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:37.355427 21303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:37.357295 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21236
I20260621 07:02:37.357473 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:37.358064 3031 external_mini_cluster.cc:1428] Setting key 443b179efdaebb1d383b4eadf730de70
I20260621 07:02:37.370138 21303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: Bootstrap starting.
Jun 21 07:02:37 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:37.373940 21303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:37.375133 21303 log.cc:824] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:37.377889 21303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: No bootstrap required, opened a new log
I20260621 07:02:37.382079 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:37.360051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.2.245.254:46835)
0621 07:02:37.360907 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:37.360919 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:37.360969 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:02:37.362546 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:37.362572 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:37.363075 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:37.363418 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:37.365114 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.366404 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.368350 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.369117 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.369255 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:37.374136 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:37.374145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:37.374163 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:37.374205 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:37.377418 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:37.378154 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:37.378163 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:37.378171 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:37.378395 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:37.378787 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:37.378791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:37.378794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:37.379511 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:37.379894 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:37.380222 (+ 328us) server_negotiation.cc:299] Negotiation successful
0621 07:02:37.380686 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":414,"threads_started":1}
I20260621 07:02:37.386709 21303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } }
I20260621 07:02:37.387074 21303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:37.387149 21303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e81dcb731643019701e47b198813a1, State: Initialized, Role: FOLLOWER
I20260621 07:02:37.387733 21303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } }
I20260621 07:02:37.387974 21303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:37.388154 21303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:37.388339 21303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:37.390407 21303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } }
I20260621 07:02:37.390903 21303 leader_election.cc:302] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25e81dcb731643019701e47b198813a1; no voters:
I20260621 07:02:37.391572 21303 leader_election.cc:288] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:37.391678 21308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:37.392413 21308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [term 1 LEADER]: Becoming Leader. State: Replica: 25e81dcb731643019701e47b198813a1, State: Running, Role: LEADER
I20260621 07:02:37.392956 21308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } }
I20260621 07:02:37.393585 21303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:37.396028 21309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25e81dcb731643019701e47b198813a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } } }
I20260621 07:02:37.396003 21310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e81dcb731643019701e47b198813a1. Latest consensus state: current_term: 1 leader_uuid: "25e81dcb731643019701e47b198813a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e81dcb731643019701e47b198813a1" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46835 } } }
I20260621 07:02:37.396550 21309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:37.396589 21310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:37.400532 21314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:37.407040 21314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:37.415620 21314 catalog_manager.cc:1347] Generated new cluster ID: e061ca44489f43b3bc70c44eb1251bd1
I20260621 07:02:37.415750 21314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:37.427467 21314 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:37.429167 21314 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:37.441471 21314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: Generated new TSK 0
I20260621 07:02:37.442504 21314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:02:37.514431 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:37.849392 21329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:37.849659 21329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:37.849838 21329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:37.857060 21329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:37.857196 21329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:37.857388 21329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:37.867139 21329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:37.868408 21329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:37.869817 21329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:37.882489 21337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:37.882511 21334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:37.883158 21335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:37.883893 21329 server_base.cc:1034] running on GCE node
I20260621 07:02:37.884433 21329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:37.885334 21329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:37.886554 21329 hybrid_clock.cc:648] HybridClock initialized: now 1782025357886502 us; error 38 us; skew 500 ppm
I20260621 07:02:37.886915 21329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:37 dist-test-slave-9zdv krb5kdc[21223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025357, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:37.890146 21329 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:02:37.891582 21329 webserver.cc:460] Webserver started at http://127.2.245.193:42745/ using document root <none> and password file <none>
I20260621 07:02:37.892225 21329 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:37.892324 21329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:37.892575 21329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:37.895494 21329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "ef477a0272614b458918d6d5f84b3230"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "2c9b54180cced62da15bca12ac02884d"
server_key_iv: "955d91f7e45f736a92c9cbaa422d6c97"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.896230 21329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "ef477a0272614b458918d6d5f84b3230"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "2c9b54180cced62da15bca12ac02884d"
server_key_iv: "955d91f7e45f736a92c9cbaa422d6c97"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.901465 21329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:02:37.905264 21344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:37.906597 21329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:37.906785 21329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "ef477a0272614b458918d6d5f84b3230"
format_stamp: "Formatted at 2026-06-21 07:02:37 on dist-test-slave-9zdv"
server_key: "2c9b54180cced62da15bca12ac02884d"
server_key_iv: "955d91f7e45f736a92c9cbaa422d6c97"
server_key_version: "encryptionkey@0"
I20260621 07:02:37.907049 21329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:37.923157 21329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:37.927383 21329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:37.927662 21329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:37.928586 21329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:37.929869 21329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:37.929941 21329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:37.930092 21329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:37.930157 21329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:37.963171 21329 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:36275
I20260621 07:02:37.963205 21456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:36275 every 8 connection(s)
I20260621 07:02:37.964476 21329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:37.971607 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21329
I20260621 07:02:37.971805 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:37.972268 3031 external_mini_cluster.cc:1428] Setting key 06b17e3226e4fc078b71e0388628a267
Jun 21 07:02:37 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025357, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:37.986012 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:37.967527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:54769 (local address 127.2.245.254:46835)
0621 07:02:37.968010 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:37.968015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:37.968821 (+ 806us) server_negotiation.cc:407] Connection header received
0621 07:02:37.970742 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:37.970748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:37.970843 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:37.971029 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:37.972803 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.973677 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.975307 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.975716 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.975855 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:37.979297 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:37.979302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:37.979305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:37.979337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:37.982601 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:37.983421 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:37.983429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:37.983436 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:37.983580 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:37.984025 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:37.984033 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:37.984038 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:37.984395 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:37.984543 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:37.985257 (+ 714us) server_negotiation.cc:299] Negotiation successful
0621 07:02:37.985514 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":275,"threads_started":1}
I20260621 07:02:37.986578 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:37.967980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46835 (local address 127.2.245.193:54769)
0621 07:02:37.968639 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:37.968684 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:37.970455 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:37.971297 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:37.971307 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:37.971891 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:37.972595 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:37.972615 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.973864 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.973868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:37.975157 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:37.975169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:37.975960 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:37.975968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:37.976192 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:37.976885 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:37.976907 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:37.979077 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:37.982751 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:37.982758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:37.982770 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:37.983290 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:37.983687 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:37.983690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:37.983692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:37.983899 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:37.984510 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:37.984515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:37.984834 (+ 319us) client_negotiation.cc:769] Sending connection context
0621 07:02:37.985049 (+ 215us) client_negotiation.cc:240] Negotiation successful
0621 07:02:37.985465 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":267,"threads_started":1}
I20260621 07:02:37.988150 21457 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46835
I20260621 07:02:37.988437 21457 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:37.989144 21457 heartbeater.cc:511] Master 127.2.245.254:46835 requested a full tablet report, sending...
I20260621 07:02:37.991310 21268 ts_manager.cc:194] Registered new tserver with Master: ef477a0272614b458918d6d5f84b3230 (127.2.245.193:36275)
I20260621 07:02:37.992753 21268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:54769
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 07:02:38.027330 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:38.349548 21463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:38.349814 21463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:38.349972 21463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:38.357757 21463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:38.357862 21463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:38.358036 21463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:38.367825 21463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:38.369091 21463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:38.370383 21463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:38.383354 21469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:38.383410 21468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:38.383388 21471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:38.384490 21463 server_base.cc:1034] running on GCE node
I20260621 07:02:38.385070 21463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:38.385953 21463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:38.387140 21463 hybrid_clock.cc:648] HybridClock initialized: now 1782025358387077 us; error 66 us; skew 500 ppm
I20260621 07:02:38.387444 21463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:38 dist-test-slave-9zdv krb5kdc[21223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:38.393651 21463 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:02:38.395218 21463 webserver.cc:460] Webserver started at http://127.2.245.194:36785/ using document root <none> and password file <none>
I20260621 07:02:38.395903 21463 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:38.396039 21463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:38.396368 21463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:38.399278 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "69c090b5f7ec4554ba2b38b224d08f8d"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "77c88ece4352cd7c4548a84220001e0b"
server_key_iv: "f37bb583959119022f3009e71d64f07b"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.400137 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "69c090b5f7ec4554ba2b38b224d08f8d"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "77c88ece4352cd7c4548a84220001e0b"
server_key_iv: "f37bb583959119022f3009e71d64f07b"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.405329 21463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:02:38.408901 21478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.410463 21463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:38.410624 21463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "69c090b5f7ec4554ba2b38b224d08f8d"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "77c88ece4352cd7c4548a84220001e0b"
server_key_iv: "f37bb583959119022f3009e71d64f07b"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.410827 21463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:38.435464 21463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:38.439760 21463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:38.440083 21463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:38.441056 21463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:38.442481 21463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:38.442554 21463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.442634 21463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:38.442677 21463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.476145 21463 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:40963
I20260621 07:02:38.476156 21590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:40963 every 8 connection(s)
I20260621 07:02:38.477465 21463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:38.481192 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21463
I20260621 07:02:38.481402 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:38.481992 3031 external_mini_cluster.cc:1428] Setting key 5de2a4e46978e7566f6282680a2a3421
Jun 21 07:02:38 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:38.501353 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:38.480054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:47179 (local address 127.2.245.254:46835)
0621 07:02:38.480273 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:38.480312 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:38.481882 (+ 1570us) server_negotiation.cc:407] Connection header received
0621 07:02:38.484592 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:38.484596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:38.484659 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:38.484746 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:38.487081 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:38.487782 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:38.490255 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:38.490711 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:38.490820 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:38.495331 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:38.495336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:38.495339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:38.495373 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:38.498118 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:38.498896 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:38.498901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:38.498903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:38.499094 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:38.499568 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:38.499573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:38.499577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:38.499863 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:38.500016 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:38.500762 (+ 746us) server_negotiation.cc:299] Negotiation successful
0621 07:02:38.500992 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 07:02:38.502440 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:38.480688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46835 (local address 127.2.245.194:47179)
0621 07:02:38.481583 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:38.481640 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:38.484284 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:38.485029 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:38.485041 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:38.485951 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:38.486831 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:38.486857 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:38.487939 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:38.487947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:38.490081 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:38.490099 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:38.490985 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:38.490994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:38.491326 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:38.492074 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:38.492107 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:38.495082 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:38.498293 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:38.498302 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:38.498322 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:38.498759 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:38.499242 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:38.499247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:38.499251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:38.499446 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:38.499997 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:38.500004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:38.500391 (+ 387us) client_negotiation.cc:769] Sending connection context
0621 07:02:38.500648 (+ 257us) client_negotiation.cc:240] Negotiation successful
0621 07:02:38.501191 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":347,"threads_started":1}
I20260621 07:02:38.503978 21591 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46835
I20260621 07:02:38.504470 21591 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:38.505439 21591 heartbeater.cc:511] Master 127.2.245.254:46835 requested a full tablet report, sending...
I20260621 07:02:38.507292 21268 ts_manager.cc:194] Registered new tserver with Master: 69c090b5f7ec4554ba2b38b224d08f8d (127.2.245.194:40963)
I20260621 07:02:38.508517 21268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:47179
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:02:38.542217 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:38.869026 21596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:38.869300 21596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:38.869480 21596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:38.876801 21596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:38.876906 21596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:38.877106 21596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:38.887135 21596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:38.888374 21596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:38.889608 21596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:38.902814 21601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:38.902824 21604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:38.902815 21602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:38.903791 21596 server_base.cc:1034] running on GCE node
I20260621 07:02:38.904377 21596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:38.905411 21596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:38.906664 21596 hybrid_clock.cc:648] HybridClock initialized: now 1782025358906597 us; error 55 us; skew 500 ppm
I20260621 07:02:38.907094 21596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:38 dist-test-slave-9zdv krb5kdc[21223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:38.910285 21596 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:02:38.911792 21596 webserver.cc:460] Webserver started at http://127.2.245.195:43213/ using document root <none> and password file <none>
I20260621 07:02:38.912510 21596 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:38.912644 21596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:38.912966 21596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:38.916411 21596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "b6ab9c131abc4a6d808767bd441eaf69"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "3434c808a715f8363e97bda94c7840a4"
server_key_iv: "032c28f7b80e6e9d8727c49d7670d44e"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.917244 21596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "b6ab9c131abc4a6d808767bd441eaf69"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "3434c808a715f8363e97bda94c7840a4"
server_key_iv: "032c28f7b80e6e9d8727c49d7670d44e"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.922641 21596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 07:02:38.926321 21611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.927683 21596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:38.927881 21596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "b6ab9c131abc4a6d808767bd441eaf69"
format_stamp: "Formatted at 2026-06-21 07:02:38 on dist-test-slave-9zdv"
server_key: "3434c808a715f8363e97bda94c7840a4"
server_key_iv: "032c28f7b80e6e9d8727c49d7670d44e"
server_key_version: "encryptionkey@0"
I20260621 07:02:38.928082 21596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:38.949087 21596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:38.953531 21596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:38.953858 21596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:38.954870 21596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:38.956239 21596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:38.956312 21596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.956475 21596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:38.956527 21596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:38.990628 21596 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:37805
I20260621 07:02:38.990674 21723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:37805 every 8 connection(s)
I20260621 07:02:38.992008 21596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:38.995875 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21596
I20260621 07:02:38.996063 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:38.996028 21457 heartbeater.cc:503] Master 127.2.245.254:46835 was elected leader, sending a full tablet report...
I20260621 07:02:38.996624 3031 external_mini_cluster.cc:1428] Setting key 1e1ee2228d3fd21c14bd978366526a8e
Jun 21 07:02:39 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:39.013064 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:38.994693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50065 (local address 127.2.245.254:46835)
0621 07:02:38.994963 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:38.994969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:38.996384 (+ 1415us) server_negotiation.cc:407] Connection header received
0621 07:02:38.998811 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:38.998816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:38.998912 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:38.999057 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:39.000806 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.001529 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.003215 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.003671 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.003807 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.007151 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:39.007156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:39.007158 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:39.007189 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:39.009884 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.010720 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.010725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.010728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.010885 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.011454 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.011459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.011463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.011725 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:39.011891 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.012587 (+ 696us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.012749 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 07:02:39.014060 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:38.995368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46835 (local address 127.2.245.195:50065)
0621 07:02:38.996209 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:38.996267 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:38.998519 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:38.999354 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:38.999365 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.000083 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:39.000656 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.000673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.001663 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.001667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.003090 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.003102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.003886 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.003892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.004095 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.004699 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:39.004724 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:39.006905 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:39.010012 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:39.010048 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:39.010060 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:39.010585 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:39.011041 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:39.011045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:39.011049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:39.011330 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:39.011866 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:39.011872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:39.012253 (+ 381us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.012476 (+ 223us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.012900 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":308,"threads_started":1}
I20260621 07:02:39.015664 21724 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46835
I20260621 07:02:39.015950 21724 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:39.016613 21724 heartbeater.cc:511] Master 127.2.245.254:46835 requested a full tablet report, sending...
I20260621 07:02:39.017982 21267 ts_manager.cc:194] Registered new tserver with Master: b6ab9c131abc4a6d808767bd441eaf69 (127.2.245.195:37805)
I20260621 07:02:39.018798 21267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:50065
I20260621 07:02:39.024547 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:39.041883 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.028664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.2.245.254:46835)
0621 07:02:39.028863 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.028867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.029315 (+ 448us) server_negotiation.cc:407] Connection header received
0621 07:02:39.029441 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.029444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.029503 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.029586 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:39.031282 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.032259 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.033625 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.034042 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.034140 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.036736 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:39.036742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:39.036747 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:39.036778 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:39.039392 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.040085 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.040088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.040091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.040213 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.040575 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.040578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.040580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.040849 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:39.041038 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.041433 (+ 395us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.041598 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 07:02:39.046078 21267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:39.052718 21267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:39.090126 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.064326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:40963 (local address 127.0.0.1:33378)
0621 07:02:39.077671 (+ 13345us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:39.077738 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:39.078011 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:39.081264 (+ 3253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:39.081272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.081310 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:39.081759 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.081782 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.084402 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.084411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.087037 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.087052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.089153 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.089157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.089323 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.089362 (+ 39us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.089523 (+ 161us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.089739 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12971,"thread_start_us":12772,"threads_started":1}
I20260621 07:02:39.090101 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.064527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.2.245.194:40963)
0621 07:02:39.080409 (+ 15882us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.080423 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.080464 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:02:39.080605 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.080617 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.080982 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.081234 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:39.082028 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.084221 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.087239 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.088745 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.088871 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.089035 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.089561 (+ 526us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.089739 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15658,"thread_start_us":16910,"threads_started":1}
I20260621 07:02:39.093468 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.064969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36275 (local address 127.0.0.1:45990)
0621 07:02:39.085217 (+ 20248us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:39.085234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:39.085386 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:39.086011 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:39.086016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.086039 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:39.086666 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.086676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.088852 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.088857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.091045 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.091066 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.092542 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.092548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.092712 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.092748 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.092870 (+ 122us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.092999 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20111,"mutex_wait_us":65,"spinlock_wait_cycles":2432,"thread_start_us":19038,"threads_started":1}
I20260621 07:02:39.093464 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.065270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.2.245.193:36275)
0621 07:02:39.083849 (+ 18579us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.083863 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.085353 (+ 1490us) server_negotiation.cc:407] Connection header received
0621 07:02:39.085522 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.085537 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.085834 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.086022 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:39.086850 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.088701 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.091236 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.092330 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.092487 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.092726 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.092910 (+ 184us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.093106 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18358,"thread_start_us":19263,"threads_started":1}
I20260621 07:02:39.094419 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.065270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.2.245.195:37805)
0621 07:02:39.085137 (+ 19867us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.085147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.085184 (+ 37us) server_negotiation.cc:407] Connection header received
0621 07:02:39.085288 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.085296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.085559 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.085818 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:39.086886 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.088447 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.091262 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.092640 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.092780 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.092989 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.093938 (+ 949us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.094110 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19291,"thread_start_us":19366,"threads_started":1}
I20260621 07:02:39.094905 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.064693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:37805 (local address 127.0.0.1:44416)
0621 07:02:39.084301 (+ 19608us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:39.084321 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:39.084441 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:39.086071 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:39.086076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.086106 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:39.086638 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.086654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.088592 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.088598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.091090 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.091102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.093156 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.093162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.093334 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.093364 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.093462 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.093585 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19457,"mutex_wait_us":4,"thread_start_us":20245,"threads_started":1}
I20260621 07:02:39.094676 21526 tablet_service.cc:1467] Processing CreateTablet for tablet ab53e511494d458e8ed550fa5bce2396 (DEFAULT_TABLE table=test-table [id=fb7c2498096f47f8b513b4e5c769ab25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:39.096668 21526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53e511494d458e8ed550fa5bce2396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:39.097304 21392 tablet_service.cc:1467] Processing CreateTablet for tablet ab53e511494d458e8ed550fa5bce2396 (DEFAULT_TABLE table=test-table [id=fb7c2498096f47f8b513b4e5c769ab25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:39.098717 21392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53e511494d458e8ed550fa5bce2396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:39.098580 21659 tablet_service.cc:1467] Processing CreateTablet for tablet ab53e511494d458e8ed550fa5bce2396 (DEFAULT_TABLE table=test-table [id=fb7c2498096f47f8b513b4e5c769ab25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:39.099967 21659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53e511494d458e8ed550fa5bce2396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:39.109433 21742 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Bootstrap starting.
I20260621 07:02:39.113013 21742 tablet_bootstrap.cc:654] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:39.113468 21743 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Bootstrap starting.
I20260621 07:02:39.115274 21742 log.cc:824] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:39.118733 21742 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: No bootstrap required, opened a new log
I20260621 07:02:39.118733 21743 tablet_bootstrap.cc:654] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:39.119069 21742 ts_tablet_manager.cc:1397] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260621 07:02:39.119089 21744 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Bootstrap starting.
I20260621 07:02:39.120041 21743 log.cc:824] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:39.122201 21744 tablet_bootstrap.cc:654] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:39.123337 21743 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: No bootstrap required, opened a new log
I20260621 07:02:39.123499 21744 log.cc:824] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:39.124361 21743 ts_tablet_manager.cc:1397] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260621 07:02:39.126381 21744 tablet_bootstrap.cc:492] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: No bootstrap required, opened a new log
I20260621 07:02:39.126657 21744 ts_tablet_manager.cc:1397] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260621 07:02:39.129489 21742 raft_consensus.cc:348] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.129967 21742 raft_consensus.cc:374] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:39.130067 21742 raft_consensus.cc:729] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c090b5f7ec4554ba2b38b224d08f8d, State: Initialized, Role: FOLLOWER
I20260621 07:02:39.130709 21742 consensus_queue.cc:260] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.131937 21591 heartbeater.cc:503] Master 127.2.245.254:46835 was elected leader, sending a full tablet report...
I20260621 07:02:39.132504 21742 ts_tablet_manager.cc:1428] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260621 07:02:39.134908 21743 raft_consensus.cc:348] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.135416 21743 raft_consensus.cc:374] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:39.135545 21743 raft_consensus.cc:729] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef477a0272614b458918d6d5f84b3230, State: Initialized, Role: FOLLOWER
I20260621 07:02:39.135536 21744 raft_consensus.cc:348] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.135958 21744 raft_consensus.cc:374] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:39.136063 21744 raft_consensus.cc:729] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ab9c131abc4a6d808767bd441eaf69, State: Initialized, Role: FOLLOWER
I20260621 07:02:39.136183 21743 consensus_queue.cc:260] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.136727 21744 consensus_queue.cc:260] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.138063 21724 heartbeater.cc:503] Master 127.2.245.254:46835 was elected leader, sending a full tablet report...
I20260621 07:02:39.138756 21744 ts_tablet_manager.cc:1428] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260621 07:02:39.138809 21750 raft_consensus.cc:482] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:39.139194 21750 raft_consensus.cc:504] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.139405 21743 ts_tablet_manager.cc:1428] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260621 07:02:39.143934 21750 leader_election.cc:288] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69c090b5f7ec4554ba2b38b224d08f8d (127.2.245.194:40963), ef477a0272614b458918d6d5f84b3230 (127.2.245.193:36275)
I20260621 07:02:39.147661 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.141611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:40963 (local address 127.2.245.195:33541)
0621 07:02:39.142154 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:39.142174 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:39.142298 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:39.142778 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:39.142783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.142809 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:39.143079 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.143092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.144451 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.144458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.145774 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.145786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.146882 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.146888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.147110 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.147134 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.147250 (+ 116us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.147381 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394}
I20260621 07:02:39.147661 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.142106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:33541 (local address 127.2.245.194:40963)
0621 07:02:39.142338 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.142345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.142365 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:02:39.142439 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.142444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.142524 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.142610 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:39.143275 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.144324 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.145937 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.146727 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.146891 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.147008 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.147279 (+ 271us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.147405 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260621 07:02:39.148566 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.142531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:50573 (local address 127.2.245.193:36275)
0621 07:02:39.142856 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.142862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.142879 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:02:39.142964 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.142972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.143045 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.143132 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:39.144199 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.144871 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.146534 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.147552 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.147766 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.147841 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.148168 (+ 327us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.148317 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260621 07:02:39.148814 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53e511494d458e8ed550fa5bce2396" candidate_uuid: "b6ab9c131abc4a6d808767bd441eaf69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" is_pre_election: true
I20260621 07:02:39.149235 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.141757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:36275 (local address 127.2.245.195:50573)
0621 07:02:39.142413 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:39.142435 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:39.142541 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:39.143430 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:39.143434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:39.143461 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:39.143722 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.143730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.145025 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.145030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.146380 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:39.146392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.147795 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.147801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:39.148001 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.148029 (+ 28us) client_negotiation.cc:769] Sending connection context
0621 07:02:39.148744 (+ 715us) client_negotiation.cc:240] Negotiation successful
0621 07:02:39.148886 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":1458,"threads_started":1}
I20260621 07:02:39.149386 21546 raft_consensus.cc:2454] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ab9c131abc4a6d808767bd441eaf69 in term 0.
I20260621 07:02:39.150159 21613 leader_election.cc:302] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c090b5f7ec4554ba2b38b224d08f8d, b6ab9c131abc4a6d808767bd441eaf69; no voters:
I20260621 07:02:39.150243 21412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53e511494d458e8ed550fa5bce2396" candidate_uuid: "b6ab9c131abc4a6d808767bd441eaf69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef477a0272614b458918d6d5f84b3230" is_pre_election: true
I20260621 07:02:39.150584 21750 raft_consensus.cc:2781] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:39.150756 21750 raft_consensus.cc:482] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:39.150861 21750 raft_consensus.cc:3037] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:39.150786 21412 raft_consensus.cc:2454] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ab9c131abc4a6d808767bd441eaf69 in term 0.
I20260621 07:02:39.153198 21750 raft_consensus.cc:504] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.153888 21750 leader_election.cc:288] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [CANDIDATE]: Term 1 election: Requested vote from peers 69c090b5f7ec4554ba2b38b224d08f8d (127.2.245.194:40963), ef477a0272614b458918d6d5f84b3230 (127.2.245.193:36275)
I20260621 07:02:39.154330 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53e511494d458e8ed550fa5bce2396" candidate_uuid: "b6ab9c131abc4a6d808767bd441eaf69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c090b5f7ec4554ba2b38b224d08f8d"
I20260621 07:02:39.154404 21412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53e511494d458e8ed550fa5bce2396" candidate_uuid: "b6ab9c131abc4a6d808767bd441eaf69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef477a0272614b458918d6d5f84b3230"
I20260621 07:02:39.154537 21546 raft_consensus.cc:3037] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:39.154623 21412 raft_consensus.cc:3037] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:39.156862 21412 raft_consensus.cc:2454] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ab9c131abc4a6d808767bd441eaf69 in term 1.
I20260621 07:02:39.156869 21546 raft_consensus.cc:2454] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ab9c131abc4a6d808767bd441eaf69 in term 1.
I20260621 07:02:39.157272 21614 leader_election.cc:302] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6ab9c131abc4a6d808767bd441eaf69, ef477a0272614b458918d6d5f84b3230; no voters:
I20260621 07:02:39.157593 21750 raft_consensus.cc:2781] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:39.158033 21750 raft_consensus.cc:686] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 1 LEADER]: Becoming Leader. State: Replica: b6ab9c131abc4a6d808767bd441eaf69, State: Running, Role: LEADER
I20260621 07:02:39.158583 21750 consensus_queue.cc:237] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } }
I20260621 07:02:39.163035 21266 catalog_manager.cc:5495] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ab9c131abc4a6d808767bd441eaf69 (127.2.245.195). New cstate: current_term: 1 leader_uuid: "b6ab9c131abc4a6d808767bd441eaf69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ab9c131abc4a6d808767bd441eaf69" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 37805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 } health_report { overall_health: UNKNOWN } } }
I20260621 07:02:39.196458 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.190768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.2.245.195:37805)
0621 07:02:39.191036 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.191042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.191061 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:02:39.191282 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.191286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.191362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.191520 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:39.191929 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.192642 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.194115 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.194610 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.194756 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.195363 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:39.195851 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:39.196033 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.196175 (+ 142us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.196229 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260621 07:02:39.204717 21412 raft_consensus.cc:1261] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 1 FOLLOWER]: Refusing update from remote peer b6ab9c131abc4a6d808767bd441eaf69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:39.204773 21546 raft_consensus.cc:1261] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 1 FOLLOWER]: Refusing update from remote peer b6ab9c131abc4a6d808767bd441eaf69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:39.205664 21752 consensus_queue.cc:1035] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c090b5f7ec4554ba2b38b224d08f8d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:39.206005 21750 consensus_queue.cc:1035] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef477a0272614b458918d6d5f84b3230" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 36275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:39.217372 21755 mvcc.cc:205] Tried to move back new op lower bound from 7299175871292346368 to 7299175871117733888. Current Snapshot: MvccSnapshot[applied={T|T < 7299175871292346368}]
W20260621 07:02:39.218716 21458 tablet.cc:2369] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:39.222747 21757 mvcc.cc:205] Tried to move back new op lower bound from 7299175871292346368 to 7299175871117733888. Current Snapshot: MvccSnapshot[applied={T|T < 7299175871292346368}]
I20260621 07:02:39.238795 21266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:39.239161 21266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:39.242782 21266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25e81dcb731643019701e47b198813a1: Sending DeleteTablet for 3 replicas of tablet ab53e511494d458e8ed550fa5bce2396
I20260621 07:02:39.243947 21659 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53e511494d458e8ed550fa5bce2396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:39 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:44416
I20260621 07:02:39.244009 21526 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53e511494d458e8ed550fa5bce2396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:39 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:33378
I20260621 07:02:39.245414 21392 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53e511494d458e8ed550fa5bce2396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:39 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:45990
I20260621 07:02:39.251303 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21329
I20260621 07:02:39.257159 21766 tablet_replica.cc:330] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: stopping tablet replica
I20260621 07:02:39.257776 21766 raft_consensus.cc:2229] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:39.257999 21767 tablet_replica.cc:330] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: stopping tablet replica
I20260621 07:02:39.258483 21766 raft_consensus.cc:2258] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:39.258471 21767 raft_consensus.cc:2229] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:39.258992 21767 raft_consensus.cc:2258] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:39.260957 21766 ts_tablet_manager.cc:1905] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:39.263125 21765 tablet_replica.cc:330] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: stopping tablet replica
I20260621 07:02:39.263624 21765 raft_consensus.cc:2229] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:39.264348 21767 ts_tablet_manager.cc:1905] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:39.264530 21765 raft_consensus.cc:2258] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:39.270076 21765 ts_tablet_manager.cc:1905] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:39.270332 21766 ts_tablet_manager.cc:1918] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:39.270522 21766 log.cc:1196] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/ab53e511494d458e8ed550fa5bce2396
I20260621 07:02:39.271085 21766 ts_tablet_manager.cc:1939] T ab53e511494d458e8ed550fa5bce2396 P 69c090b5f7ec4554ba2b38b224d08f8d: Deleting consensus metadata
I20260621 07:02:39.271843 21767 ts_tablet_manager.cc:1918] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:39.272027 21767 log.cc:1196] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/ab53e511494d458e8ed550fa5bce2396
I20260621 07:02:39.272569 21767 ts_tablet_manager.cc:1939] T ab53e511494d458e8ed550fa5bce2396 P ef477a0272614b458918d6d5f84b3230: Deleting consensus metadata
Jun 21 07:02:39 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:02:39.273252 21253 catalog_manager.cc:4841] TS 69c090b5f7ec4554ba2b38b224d08f8d (127.2.245.194:40963): tablet ab53e511494d458e8ed550fa5bce2396 (table test-table [id=fb7c2498096f47f8b513b4e5c769ab25]) successfully deleted
I20260621 07:02:39.274474 21254 catalog_manager.cc:4841] TS ef477a0272614b458918d6d5f84b3230 (127.2.245.193:36275): tablet ab53e511494d458e8ed550fa5bce2396 (table test-table [id=fb7c2498096f47f8b513b4e5c769ab25]) successfully deleted
I20260621 07:02:39.278859 21765 ts_tablet_manager.cc:1918] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:39.279104 21765 log.cc:1196] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/ab53e511494d458e8ed550fa5bce2396
I20260621 07:02:39.279665 21765 ts_tablet_manager.cc:1939] T ab53e511494d458e8ed550fa5bce2396 P b6ab9c131abc4a6d808767bd441eaf69: Deleting consensus metadata
I20260621 07:02:39.281488 21255 catalog_manager.cc:4841] TS b6ab9c131abc4a6d808767bd441eaf69 (127.2.245.195:37805): tablet ab53e511494d458e8ed550fa5bce2396 (table test-table [id=fb7c2498096f47f8b513b4e5c769ab25]) successfully deleted
I20260621 07:02:39.282239 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.253215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.2.245.193:36275)
0621 07:02:39.253447 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.253456 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.262356 (+ 8900us) server_negotiation.cc:407] Connection header received
0621 07:02:39.262560 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.262565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.262651 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.262783 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:39.264648 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.265869 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.267866 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.268567 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.268682 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.274538 (+ 5856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:39.274553 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:39.274571 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:39.274624 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:39.278634 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.279664 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.279675 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.279679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.279866 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.280405 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.280410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.280413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.280924 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:39.281380 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.281552 (+ 172us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.281863 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:02:39.282932 21452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:39.383955 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21329
I20260621 07:02:39.402566 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21463
Jun 21 07:02:39 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:02:39.420159 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.403509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.2.245.194:40963)
0621 07:02:39.403714 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.403721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.403752 (+ 31us) server_negotiation.cc:407] Connection header received
0621 07:02:39.403850 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.403857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.403949 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.404063 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:39.405254 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.406438 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.407851 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.408482 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.408616 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.413010 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:39.413028 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:39.413048 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:39.413093 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:39.417118 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.417877 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.417886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.417889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.418045 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.418430 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.418435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.418439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.418987 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:39.419284 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.419550 (+ 266us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.419788 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:02:39.420863 21586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:39.512183 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21463
I20260621 07:02:39.535838 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21596
Jun 21 07:02:39 dist-test-slave-9zdv krb5kdc[21223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:02:39.552707 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:39.536720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.2.245.195:37805)
0621 07:02:39.536952 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:39.536958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:39.536978 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:02:39.537109 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:39.537121 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:39.537190 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:39.537313 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:39.538696 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.539647 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.540973 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:39.541450 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:39.541633 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:39.546477 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:39.546490 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:39.546507 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:39.546543 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:39.549731 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.550468 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.550476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.550478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.550624 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:39.551058 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:39.551063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:39.551066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:39.551458 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:39.551797 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:39.552095 (+ 298us) server_negotiation.cc:299] Negotiation successful
0621 07:02:39.552334 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260621 07:02:39.553562 21719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:39.677847 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21596
I20260621 07:02:39.701628 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21236
I20260621 07:02:39.702837 21298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:39.815277 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21236
2026-06-21T07:02:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4250 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-21T07:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:39Z Disabled control of system clock
I20260621 07:02:39.890704 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44019
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:40.220999 21776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:40.221294 21776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:40.221428 21776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:40.228945 21776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:40.229071 21776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:40.229110 21776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:40.229231 21776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:40.229312 21776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:40.238534 21776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:40.239959 21776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:40.241804 21776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:40.253460 21781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:40.253557 21782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:40.253438 21784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:40.254550 21776 server_base.cc:1034] running on GCE node
I20260621 07:02:40.255438 21776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:40.256879 21776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:40.258073 21776 hybrid_clock.cc:648] HybridClock initialized: now 1782025360258038 us; error 29 us; skew 500 ppm
I20260621 07:02:40.258458 21776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:40.260730 21776 webserver.cc:460] Webserver started at http://127.2.245.254:45607/ using document root <none> and password file <none>
I20260621 07:02:40.261415 21776 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:40.261526 21776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:40.261878 21776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:40.264835 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "86dc72e09aaf452fba20670340df126b"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "d101a5baf43f34fcca4da41c5e33d5b3"
server_key_iv: "43366a0320e99b2f79107e10379d7c04"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.265628 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "86dc72e09aaf452fba20670340df126b"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "d101a5baf43f34fcca4da41c5e33d5b3"
server_key_iv: "43366a0320e99b2f79107e10379d7c04"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.270701 21776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260621 07:02:40.274084 21790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:40.275454 21776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:40.275645 21776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "86dc72e09aaf452fba20670340df126b"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "d101a5baf43f34fcca4da41c5e33d5b3"
server_key_iv: "43366a0320e99b2f79107e10379d7c04"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.275861 21776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:40.303918 21776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:40.304823 21776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:40.305126 21776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:40.324052 21776 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:44019
I20260621 07:02:40.324079 21841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:44019 every 8 connection(s)
I20260621 07:02:40.325454 21776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:40.329324 21842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:40.333820 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21776
I20260621 07:02:40.334033 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:40.334471 3031 external_mini_cluster.cc:1428] Setting key fb2b8f90de151ed6e0678e367419ff99
I20260621 07:02:40.342383 21842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: Bootstrap starting.
I20260621 07:02:40.346139 21842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:40.347550 21842 log.cc:824] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:40.351033 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:40.336609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.2.245.254:44019)
0621 07:02:40.337784 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:40.337807 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:40.337869 (+ 62us) server_negotiation.cc:407] Connection header received
0621 07:02:40.340492 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:40.340525 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:40.341186 (+ 661us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:40.341653 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:40.342273 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.344280 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.345626 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.346786 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.347002 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:40.348227 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:40.348238 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:40.348258 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:40.348911 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:40.349013 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:40.349042 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:40.349167 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:40.349385 (+ 218us) server_negotiation.cc:299] Negotiation successful
0621 07:02:40.349709 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":356,"threads_started":1}
I20260621 07:02:40.351074 21842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: No bootstrap required, opened a new log
I20260621 07:02:40.359392 21842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } }
I20260621 07:02:40.359764 21842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:40.359831 21842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dc72e09aaf452fba20670340df126b, State: Initialized, Role: FOLLOWER
I20260621 07:02:40.360435 21842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } }
I20260621 07:02:40.360680 21842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:40.360786 21842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:40.360920 21842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:40.363206 21842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } }
I20260621 07:02:40.363745 21842 leader_election.cc:302] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86dc72e09aaf452fba20670340df126b; no voters:
I20260621 07:02:40.364293 21842 leader_election.cc:288] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:40.364470 21847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:40.365666 21847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [term 1 LEADER]: Becoming Leader. State: Replica: 86dc72e09aaf452fba20670340df126b, State: Running, Role: LEADER
I20260621 07:02:40.366266 21847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } }
I20260621 07:02:40.366688 21842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:40.369448 21849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86dc72e09aaf452fba20670340df126b. Latest consensus state: current_term: 1 leader_uuid: "86dc72e09aaf452fba20670340df126b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } } }
I20260621 07:02:40.369484 21848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86dc72e09aaf452fba20670340df126b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dc72e09aaf452fba20670340df126b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44019 } } }
I20260621 07:02:40.369864 21849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:40.369917 21848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:40.372320 21855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:40.378053 21855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:40.387192 21855 catalog_manager.cc:1347] Generated new cluster ID: 20239e90234e44aca5924663f63b1243
I20260621 07:02:40.387329 21855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:40.403235 21855 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:40.404448 21855 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:40.411692 21855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: Generated new TSK 0
I20260621 07:02:40.412583 21855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:40.424526 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--builtin_ntp_servers=127.2.245.212:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:40.783339 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:40.783705 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:40.783885 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:40.791469 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:40.791613 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:40.791807 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:40.801164 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:40.802521 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:40.804418 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:40.817458 21871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:40.817657 21872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:40.817644 21874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:40.818611 21866 server_base.cc:1034] running on GCE node
I20260621 07:02:40.819391 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:40.820392 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:40.821636 21866 hybrid_clock.cc:648] HybridClock initialized: now 1782025360821597 us; error 53 us; skew 500 ppm
I20260621 07:02:40.821993 21866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:40.824370 21866 webserver.cc:460] Webserver started at http://127.2.245.193:34785/ using document root <none> and password file <none>
I20260621 07:02:40.825075 21866 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:40.825218 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:40.825565 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:40.828488 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "8a475061f65045bf99dad52082664cdf"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "88f0b1294774772e54500575d183bbaa"
server_key_iv: "49215e1bbda41ed20199af17aa1075c1"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.829276 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "8a475061f65045bf99dad52082664cdf"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "88f0b1294774772e54500575d183bbaa"
server_key_iv: "49215e1bbda41ed20199af17aa1075c1"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.834414 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:02:40.838135 21880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:40.839540 21866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 07:02:40.839715 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "8a475061f65045bf99dad52082664cdf"
format_stamp: "Formatted at 2026-06-21 07:02:40 on dist-test-slave-9zdv"
server_key: "88f0b1294774772e54500575d183bbaa"
server_key_iv: "49215e1bbda41ed20199af17aa1075c1"
server_key_version: "encryptionkey@0"
I20260621 07:02:40.839951 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:40.853485 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:40.854322 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:40.854640 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:40.855696 21866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:40.857177 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:40.857321 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:40.857493 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:40.857606 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:40.892380 21866 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:38367
I20260621 07:02:40.892431 21992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:38367 every 8 connection(s)
I20260621 07:02:40.893714 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:40.899299 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21866
I20260621 07:02:40.899493 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:40.899943 3031 external_mini_cluster.cc:1428] Setting key a2da9b036d5e5d047e7a2f5ffba99180
I20260621 07:02:40.903184 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--builtin_ntp_servers=127.2.245.212:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:40.908169 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:40.896482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:59903 (local address 127.2.245.254:44019)
0621 07:02:40.896966 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:40.896972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:40.898161 (+ 1189us) server_negotiation.cc:407] Connection header received
0621 07:02:40.900242 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:40.900248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:40.900344 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:40.900471 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:40.902010 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.903144 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.904880 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.905586 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.905729 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:40.907148 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:40.907156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:40.907164 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:40.907191 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:40.907248 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:40.907268 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:40.907363 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:40.907569 (+ 206us) server_negotiation.cc:299] Negotiation successful
0621 07:02:40.907687 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":233,"threads_started":1}
I20260621 07:02:40.909045 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:40.897045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44019 (local address 127.2.245.193:59903)
0621 07:02:40.898000 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:40.898042 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:40.899910 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:40.900799 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:40.900813 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:40.900927 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:40.901827 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:40.901858 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.903334 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.903340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:40.904686 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:40.904699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:40.905825 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:40.905830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:40.906035 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:40.906777 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:40.906780 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:40.906822 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:40.906905 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:40.907367 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:40.907375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:40.907381 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 07:02:40.907525 (+ 144us) client_negotiation.cc:240] Negotiation successful
0621 07:02:40.907874 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":317,"threads_started":1}
I20260621 07:02:40.910676 21993 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44019
I20260621 07:02:40.911053 21993 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:40.912052 21993 heartbeater.cc:511] Master 127.2.245.254:44019 requested a full tablet report, sending...
I20260621 07:02:40.914279 21807 ts_manager.cc:194] Registered new tserver with Master: 8a475061f65045bf99dad52082664cdf (127.2.245.193:38367)
I20260621 07:02:40.915794 21807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.193:59903
W20260621 07:02:41.236486 21997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:41.236825 21997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:41.236995 21997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:41.244508 21997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:41.244644 21997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:41.244827 21997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:41.254146 21997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:41.255514 21997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:41.257292 21997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:41.270606 22005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:41.270663 22002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:41.270694 22003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:41.271991 21997 server_base.cc:1034] running on GCE node
I20260621 07:02:41.272733 21997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:41.273752 21997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:41.275071 21997 hybrid_clock.cc:648] HybridClock initialized: now 1782025361275000 us; error 50 us; skew 500 ppm
I20260621 07:02:41.275444 21997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:41.277911 21997 webserver.cc:460] Webserver started at http://127.2.245.194:45585/ using document root <none> and password file <none>
I20260621 07:02:41.278661 21997 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:41.278766 21997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:41.279182 21997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:41.282109 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "9ccff4ede0b0476fb433d7cea10b407d"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "c0ff48eac697cced2962c84800b6ab8b"
server_key_iv: "84996e53ee5e39ff187c5e6777a160a7"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.282898 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "9ccff4ede0b0476fb433d7cea10b407d"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "c0ff48eac697cced2962c84800b6ab8b"
server_key_iv: "84996e53ee5e39ff187c5e6777a160a7"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.288205 21997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 07:02:41.292124 22011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.293622 21997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:41.293848 21997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "9ccff4ede0b0476fb433d7cea10b407d"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "c0ff48eac697cced2962c84800b6ab8b"
server_key_iv: "84996e53ee5e39ff187c5e6777a160a7"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.294093 21997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:41.324602 21997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:41.325532 21997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:41.325857 21997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:41.326880 21997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:41.328482 21997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:41.328608 21997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.328756 21997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:41.328852 21997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.363480 21997 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:38047
I20260621 07:02:41.363574 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:38047 every 8 connection(s)
I20260621 07:02:41.364822 21997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:41.370215 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 21997
I20260621 07:02:41.370414 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:41.370908 3031 external_mini_cluster.cc:1428] Setting key ead562c0ecbde6c70348e2622a9c81a1
I20260621 07:02:41.373998 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--builtin_ntp_servers=127.2.245.212:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:41.379290 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.367515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:53609 (local address 127.2.245.254:44019)
0621 07:02:41.367694 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.367698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.369182 (+ 1484us) server_negotiation.cc:407] Connection header received
0621 07:02:41.371170 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.371176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.371277 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.371400 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:41.372733 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.373659 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.376308 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.376773 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.376937 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.378388 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:41.378394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:41.378399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:41.378429 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:41.378461 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:41.378480 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:41.378583 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.378826 (+ 243us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.378981 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:02:41.380429 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.368131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44019 (local address 127.2.245.194:53609)
0621 07:02:41.369033 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:41.369068 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:41.370822 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:41.371658 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:41.371666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:41.371760 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:41.372519 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.372542 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.373838 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.373844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.376149 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.376168 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.377043 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.377053 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.377294 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.378043 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:41.378046 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:41.378090 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:41.378173 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:41.378589 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:41.378602 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:41.378611 (+ 9us) client_negotiation.cc:769] Sending connection context
0621 07:02:41.378802 (+ 191us) client_negotiation.cc:240] Negotiation successful
0621 07:02:41.379297 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":286,"threads_started":1}
I20260621 07:02:41.382045 22124 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44019
I20260621 07:02:41.382406 22124 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:41.383373 22124 heartbeater.cc:511] Master 127.2.245.254:44019 requested a full tablet report, sending...
I20260621 07:02:41.384933 21807 ts_manager.cc:194] Registered new tserver with Master: 9ccff4ede0b0476fb433d7cea10b407d (127.2.245.194:38047)
I20260621 07:02:41.385771 21807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.194:53609
W20260621 07:02:41.707949 22127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:41.708225 22127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:41.708374 22127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:41.715610 22127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:41.715709 22127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:41.715871 22127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:41.724990 22127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:41.726332 22127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:41.728157 22127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:41.741771 22132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:41.741856 22135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:41.741884 22133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:41.742640 22127 server_base.cc:1034] running on GCE node
I20260621 07:02:41.743355 22127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:41.744416 22127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:41.745657 22127 hybrid_clock.cc:648] HybridClock initialized: now 1782025361745609 us; error 71 us; skew 500 ppm
I20260621 07:02:41.746004 22127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:41.748368 22127 webserver.cc:460] Webserver started at http://127.2.245.195:32963/ using document root <none> and password file <none>
I20260621 07:02:41.749079 22127 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:41.749215 22127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:41.749547 22127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:41.752485 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "c25bcf039a684d89a6a006ea1361a984"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "297d99ea30058f9a48a29855460a537f"
server_key_iv: "d7ea47e527c984abc50e729de6a355bd"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.753268 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "c25bcf039a684d89a6a006ea1361a984"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "297d99ea30058f9a48a29855460a537f"
server_key_iv: "d7ea47e527c984abc50e729de6a355bd"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.758407 22127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:02:41.762017 22141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.763427 22127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:41.763612 22127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "c25bcf039a684d89a6a006ea1361a984"
format_stamp: "Formatted at 2026-06-21 07:02:41 on dist-test-slave-9zdv"
server_key: "297d99ea30058f9a48a29855460a537f"
server_key_iv: "d7ea47e527c984abc50e729de6a355bd"
server_key_version: "encryptionkey@0"
I20260621 07:02:41.763842 22127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:41.788568 22127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:41.789464 22127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:41.789781 22127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:41.790836 22127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:41.792373 22127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:41.792495 22127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.792629 22127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:41.792716 22127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:41.827198 22127 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:43207
I20260621 07:02:41.827294 22253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:43207 every 8 connection(s)
I20260621 07:02:41.828536 22127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:41.831676 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22127
I20260621 07:02:41.831871 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:41.832469 3031 external_mini_cluster.cc:1428] Setting key 0357b3c01a2fa5b06288b27f6c207955
I20260621 07:02:41.842101 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.831225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:59839 (local address 127.2.245.254:44019)
0621 07:02:41.831423 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.831427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.833202 (+ 1775us) server_negotiation.cc:407] Connection header received
0621 07:02:41.835736 (+ 2534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.835740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.835809 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.835886 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:41.836909 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.837630 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.839249 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.839669 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.839840 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.841236 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:41.841241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:41.841245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:41.841265 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:41.841297 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:41.841309 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:41.841409 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.841658 (+ 249us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.841778 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 07:02:41.843181 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.831972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44019 (local address 127.2.245.195:59839)
0621 07:02:41.833019 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:41.833076 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:41.835481 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:41.836080 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:41.836088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:41.836181 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:41.836767 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.836783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.837784 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.837788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.839087 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.839100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.839933 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.839940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.840181 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.840869 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:41.840872 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:41.840911 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:41.840993 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:41.841408 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:41.841421 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:41.841431 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 07:02:41.841630 (+ 199us) client_negotiation.cc:240] Negotiation successful
0621 07:02:41.842041 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":302,"threads_started":1}
I20260621 07:02:41.844678 22254 heartbeater.cc:347] Connected to a master server at 127.2.245.254:44019
I20260621 07:02:41.845088 22254 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:41.845816 22254 heartbeater.cc:511] Master 127.2.245.254:44019 requested a full tablet report, sending...
I20260621 07:02:41.847277 21807 ts_manager.cc:194] Registered new tserver with Master: c25bcf039a684d89a6a006ea1361a984 (127.2.245.195:43207)
I20260621 07:02:41.848261 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:41.848328 21807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.195:59839
I20260621 07:02:41.857918 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.852427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37038 (local address 127.2.245.254:44019)
0621 07:02:41.852623 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.852627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.853089 (+ 462us) server_negotiation.cc:407] Connection header received
0621 07:02:41.853215 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.853219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.853290 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.853374 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:41.853802 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.854467 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.855791 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.856219 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.856308 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.857300 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:41.857304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:41.857307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:41.857324 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:41.857354 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:41.857365 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:41.857475 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.857626 (+ 151us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.857667 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:02:41.861714 21807 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:41.868681 21807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:41.892056 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.880890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56106)
0621 07:02:41.882362 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:41.882400 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:41.882652 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:41.884021 (+ 1369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:41.884031 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:41.884067 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:41.884451 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.884465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.886811 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.886820 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.888695 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.888709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.891026 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.891035 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.891240 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.891295 (+ 55us) client_negotiation.cc:769] Sending connection context
0621 07:02:41.891442 (+ 147us) client_negotiation.cc:240] Negotiation successful
0621 07:02:41.891659 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"mutex_wait_us":23,"thread_start_us":2568,"threads_started":1}
I20260621 07:02:41.892743 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.882111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.2.245.195:43207)
0621 07:02:41.884453 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.884461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.884497 (+ 36us) server_negotiation.cc:407] Connection header received
0621 07:02:41.884593 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.884602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.884882 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.885074 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:41.885777 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.887127 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.889740 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.890952 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.891116 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.891378 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.892182 (+ 804us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.892374 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":1973,"threads_started":1}
I20260621 07:02:41.892889 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.881147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.2.245.193:38367)
0621 07:02:41.881798 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.881809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.882507 (+ 698us) server_negotiation.cc:407] Connection header received
0621 07:02:41.882918 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.882926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.883209 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.883685 (+ 476us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:41.884652 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.886529 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.888894 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.890791 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.890926 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.891221 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.892306 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.892486 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":670,"threads_started":1}
I20260621 07:02:41.893251 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.881674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:43207 (local address 127.0.0.1:36642)
0621 07:02:41.883754 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:41.883774 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:41.883882 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:41.885148 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:41.885158 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:41.885186 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:41.885592 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.885610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.887263 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.887268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.889501 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.889515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.891503 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.891509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.891687 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.891716 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 07:02:41.891798 (+ 82us) client_negotiation.cc:240] Negotiation successful
0621 07:02:41.892057 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932,"thread_start_us":390,"threads_started":1}
I20260621 07:02:41.894318 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.884621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38047 (local address 127.0.0.1:58668)
0621 07:02:41.885169 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:41.885186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:41.885403 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:41.886609 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:41.886615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:41.886653 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:41.887168 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.887181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.889166 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.889171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.891199 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:41.891213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.893475 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.893483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:41.893680 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.893727 (+ 47us) client_negotiation.cc:769] Sending connection context
0621 07:02:41.893804 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 07:02:41.893936 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":22,"thread_start_us":312,"threads_started":1}
I20260621 07:02:41.897722 22189 tablet_service.cc:1467] Processing CreateTablet for tablet 8c610b002dec4c70b08e1b90fac627bf (DEFAULT_TABLE table=test-table [id=3776ff3d0d2348a683706410c8e9879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:41.897724 21928 tablet_service.cc:1467] Processing CreateTablet for tablet 8c610b002dec4c70b08e1b90fac627bf (DEFAULT_TABLE table=test-table [id=3776ff3d0d2348a683706410c8e9879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:41.899271 22189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c610b002dec4c70b08e1b90fac627bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:41.899703 21928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c610b002dec4c70b08e1b90fac627bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:41.904146 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:41.884234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.2.245.194:38047)
0621 07:02:41.884798 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:41.884806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:41.885287 (+ 481us) server_negotiation.cc:407] Connection header received
0621 07:02:41.885623 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:41.885630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:41.885876 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:41.886376 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:41.887418 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.889010 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.891413 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:41.893255 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:41.893505 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:41.903162 (+ 9657us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:41.903276 (+ 114us) server_negotiation.cc:299] Negotiation successful
0621 07:02:41.903484 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":489,"threads_started":1}
I20260621 07:02:41.908561 22059 tablet_service.cc:1467] Processing CreateTablet for tablet 8c610b002dec4c70b08e1b90fac627bf (DEFAULT_TABLE table=test-table [id=3776ff3d0d2348a683706410c8e9879c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:41.910065 22059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c610b002dec4c70b08e1b90fac627bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:41.912613 22272 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Bootstrap starting.
I20260621 07:02:41.913444 22273 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Bootstrap starting.
I20260621 07:02:41.917133 22273 tablet_bootstrap.cc:654] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:41.918565 22272 tablet_bootstrap.cc:654] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:41.918720 21993 heartbeater.cc:503] Master 127.2.245.254:44019 was elected leader, sending a full tablet report...
I20260621 07:02:41.919682 22273 log.cc:824] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:41.920055 22272 log.cc:824] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:41.924757 22276 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Bootstrap starting.
I20260621 07:02:41.926517 22272 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: No bootstrap required, opened a new log
I20260621 07:02:41.926854 22272 ts_tablet_manager.cc:1397] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260621 07:02:41.928399 22273 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: No bootstrap required, opened a new log
I20260621 07:02:41.928742 22273 ts_tablet_manager.cc:1397] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260621 07:02:41.929385 22276 tablet_bootstrap.cc:654] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:41.930677 22276 log.cc:824] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:41.933934 22276 tablet_bootstrap.cc:492] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: No bootstrap required, opened a new log
I20260621 07:02:41.934217 22276 ts_tablet_manager.cc:1397] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260621 07:02:41.936728 22273 raft_consensus.cc:348] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.937105 22273 raft_consensus.cc:374] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:41.937238 22273 raft_consensus.cc:729] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25bcf039a684d89a6a006ea1361a984, State: Initialized, Role: FOLLOWER
I20260621 07:02:41.937227 22272 raft_consensus.cc:348] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.937759 22272 raft_consensus.cc:374] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:41.937865 22272 raft_consensus.cc:729] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a475061f65045bf99dad52082664cdf, State: Initialized, Role: FOLLOWER
I20260621 07:02:41.937929 22273 consensus_queue.cc:260] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.938576 22272 consensus_queue.cc:260] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.940493 22272 ts_tablet_manager.cc:1428] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260621 07:02:41.941505 22254 heartbeater.cc:503] Master 127.2.245.254:44019 was elected leader, sending a full tablet report...
I20260621 07:02:41.943035 22273 ts_tablet_manager.cc:1428] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260621 07:02:41.946022 22276 raft_consensus.cc:348] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.946424 22276 raft_consensus.cc:374] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:41.946556 22276 raft_consensus.cc:729] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccff4ede0b0476fb433d7cea10b407d, State: Initialized, Role: FOLLOWER
I20260621 07:02:41.947201 22276 consensus_queue.cc:260] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:41.948167 22124 heartbeater.cc:503] Master 127.2.245.254:44019 was elected leader, sending a full tablet report...
I20260621 07:02:41.948701 22276 ts_tablet_manager.cc:1428] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
W20260621 07:02:42.080391 22255 tablet.cc:2369] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:42.116885 22125 tablet.cc:2369] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:42.146246 21994 tablet.cc:2369] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:42.326014 22279 raft_consensus.cc:482] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:42.326347 22279 raft_consensus.cc:504] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:42.328293 22279 leader_election.cc:288] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c25bcf039a684d89a6a006ea1361a984 (127.2.245.195:43207), 9ccff4ede0b0476fb433d7cea10b407d (127.2.245.194:38047)
I20260621 07:02:42.336030 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.329007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:45115 (local address 127.2.245.195:43207)
0621 07:02:42.329511 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.329520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.330059 (+ 539us) server_negotiation.cc:407] Connection header received
0621 07:02:42.330178 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.330183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.330259 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.330359 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:42.331195 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.332228 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.334099 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.335088 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.335248 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.335322 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.335627 (+ 305us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.335773 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313}
I20260621 07:02:42.336081 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.328694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:43207 (local address 127.2.245.193:45115)
0621 07:02:42.329944 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.329961 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:42.330069 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:42.330504 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:42.330508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:42.330556 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:42.331005 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:42.331017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.332369 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.332375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:42.333901 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:42.333914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.335269 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.335282 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:42.335445 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.335481 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 07:02:42.335574 (+ 93us) client_negotiation.cc:240] Negotiation successful
0621 07:02:42.335701 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1144,"mutex_wait_us":25,"thread_start_us":580,"threads_started":1}
I20260621 07:02:42.336948 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.329222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:60329 (local address 127.2.245.194:38047)
0621 07:02:42.329706 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.329711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.329811 (+ 100us) server_negotiation.cc:407] Connection header received
0621 07:02:42.330152 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.330158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.330248 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.330360 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:42.330957 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.332025 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.334678 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.335780 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.335940 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.336000 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.336567 (+ 567us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.336698 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260621 07:02:42.337006 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.328668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:38047 (local address 127.2.245.193:60329)
0621 07:02:42.329667 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.329692 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:42.329944 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:42.330381 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:42.330386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:42.330408 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:42.330780 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:42.330789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.332160 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.332167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:42.334485 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:42.334499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.336146 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.336155 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:42.336374 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.336405 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 07:02:42.336504 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 07:02:42.336646 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":79,"thread_start_us":1003,"threads_started":1}
I20260621 07:02:42.337549 22209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c610b002dec4c70b08e1b90fac627bf" candidate_uuid: "8a475061f65045bf99dad52082664cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25bcf039a684d89a6a006ea1361a984" is_pre_election: true
I20260621 07:02:42.338153 22209 raft_consensus.cc:2454] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a475061f65045bf99dad52082664cdf in term 0.
I20260621 07:02:42.338020 22079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c610b002dec4c70b08e1b90fac627bf" candidate_uuid: "8a475061f65045bf99dad52082664cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ccff4ede0b0476fb433d7cea10b407d" is_pre_election: true
I20260621 07:02:42.338562 22079 raft_consensus.cc:2454] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a475061f65045bf99dad52082664cdf in term 0.
I20260621 07:02:42.338892 21882 leader_election.cc:302] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a475061f65045bf99dad52082664cdf, c25bcf039a684d89a6a006ea1361a984; no voters:
I20260621 07:02:42.339392 22279 raft_consensus.cc:2781] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:42.339551 22279 raft_consensus.cc:482] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:42.339670 22279 raft_consensus.cc:3037] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:42.341825 22279 raft_consensus.cc:504] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:42.342531 22279 leader_election.cc:288] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [CANDIDATE]: Term 1 election: Requested vote from peers c25bcf039a684d89a6a006ea1361a984 (127.2.245.195:43207), 9ccff4ede0b0476fb433d7cea10b407d (127.2.245.194:38047)
I20260621 07:02:42.343034 22209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c610b002dec4c70b08e1b90fac627bf" candidate_uuid: "8a475061f65045bf99dad52082664cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c25bcf039a684d89a6a006ea1361a984"
I20260621 07:02:42.343075 22079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c610b002dec4c70b08e1b90fac627bf" candidate_uuid: "8a475061f65045bf99dad52082664cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ccff4ede0b0476fb433d7cea10b407d"
I20260621 07:02:42.343262 22079 raft_consensus.cc:3037] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:42.343273 22209 raft_consensus.cc:3037] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:42.346542 22209 raft_consensus.cc:2454] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a475061f65045bf99dad52082664cdf in term 1.
I20260621 07:02:42.346542 22079 raft_consensus.cc:2454] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a475061f65045bf99dad52082664cdf in term 1.
I20260621 07:02:42.346979 21882 leader_election.cc:302] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a475061f65045bf99dad52082664cdf, c25bcf039a684d89a6a006ea1361a984; no voters:
I20260621 07:02:42.347314 22279 raft_consensus.cc:2781] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:42.347788 22279 raft_consensus.cc:686] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 1 LEADER]: Becoming Leader. State: Replica: 8a475061f65045bf99dad52082664cdf, State: Running, Role: LEADER
I20260621 07:02:42.348332 22279 consensus_queue.cc:237] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } }
I20260621 07:02:42.353119 21806 catalog_manager.cc:5495] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a475061f65045bf99dad52082664cdf (127.2.245.193). New cstate: current_term: 1 leader_uuid: "8a475061f65045bf99dad52082664cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a475061f65045bf99dad52082664cdf" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 } health_report { overall_health: UNKNOWN } } }
I20260621 07:02:42.468436 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.462043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56114 (local address 127.2.245.193:38367)
0621 07:02:42.462561 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.462566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.462756 (+ 190us) server_negotiation.cc:407] Connection header received
0621 07:02:42.462996 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.463001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.463098 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.463261 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:42.463716 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.464445 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.465945 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.466369 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.466480 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.467218 (+ 738us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:42.467765 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:42.467906 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.468144 (+ 238us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.468193 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":300,"threads_started":1}
I20260621 07:02:42.478987 22209 raft_consensus.cc:1261] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 1 FOLLOWER]: Refusing update from remote peer 8a475061f65045bf99dad52082664cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:42.478979 22079 raft_consensus.cc:1261] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 1 FOLLOWER]: Refusing update from remote peer 8a475061f65045bf99dad52082664cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:42.480160 22283 consensus_queue.cc:1035] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ccff4ede0b0476fb433d7cea10b407d" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:42.481330 22283 consensus_queue.cc:1035] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c25bcf039a684d89a6a006ea1361a984" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 43207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:42.493000 22288 mvcc.cc:205] Tried to move back new op lower bound from 7299175884701552640 to 7299175884182761472. Current Snapshot: MvccSnapshot[applied={T|T < 7299175884701552640}]
I20260621 07:02:42.506035 21806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:42.506284 21806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:42.509755 21806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86dc72e09aaf452fba20670340df126b: Sending DeleteTablet for 3 replicas of tablet 8c610b002dec4c70b08e1b90fac627bf
I20260621 07:02:42.510983 22189 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c610b002dec4c70b08e1b90fac627bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:42 UTC) from {username='slave'} at 127.0.0.1:36642
I20260621 07:02:42.511005 21928 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c610b002dec4c70b08e1b90fac627bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:42 UTC) from {username='slave'} at 127.0.0.1:56106
I20260621 07:02:42.511088 22059 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c610b002dec4c70b08e1b90fac627bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:42 UTC) from {username='slave'} at 127.0.0.1:58668
I20260621 07:02:42.512193 22301 tablet_replica.cc:330] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: stopping tablet replica
I20260621 07:02:42.512600 22301 raft_consensus.cc:2229] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:42.512800 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21866
I20260621 07:02:42.513163 22301 raft_consensus.cc:2258] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:42.521603 22301 ts_tablet_manager.cc:1905] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:42.538730 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.515651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.2.245.193:38367)
0621 07:02:42.515895 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.515904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.528942 (+ 13038us) server_negotiation.cc:407] Connection header received
0621 07:02:42.529609 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.529616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.529731 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.529811 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:42.530521 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.531957 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.533911 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.534632 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.534732 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.536465 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:42.536482 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:42.536502 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:42.537366 (+ 864us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:42.537429 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:42.537465 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:42.537546 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.538229 (+ 683us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.538346 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:02:42.539860 22302 tablet_replica.cc:330] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: stopping tablet replica
I20260621 07:02:42.540369 21988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:42.540850 22302 raft_consensus.cc:2229] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:42.541334 22302 raft_consensus.cc:2258] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:42.541849 22301 ts_tablet_manager.cc:1918] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:42.542078 22301 log.cc:1196] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/8c610b002dec4c70b08e1b90fac627bf
I20260621 07:02:42.542799 22301 ts_tablet_manager.cc:1939] T 8c610b002dec4c70b08e1b90fac627bf P c25bcf039a684d89a6a006ea1361a984: Deleting consensus metadata
I20260621 07:02:42.543718 22302 ts_tablet_manager.cc:1905] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:42.545020 21792 catalog_manager.cc:4841] TS c25bcf039a684d89a6a006ea1361a984 (127.2.245.195:43207): tablet 8c610b002dec4c70b08e1b90fac627bf (table test-table [id=3776ff3d0d2348a683706410c8e9879c]) successfully deleted
I20260621 07:02:42.551326 22302 ts_tablet_manager.cc:1918] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:42.551515 22302 log.cc:1196] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/8c610b002dec4c70b08e1b90fac627bf
I20260621 07:02:42.552055 22302 ts_tablet_manager.cc:1939] T 8c610b002dec4c70b08e1b90fac627bf P 9ccff4ede0b0476fb433d7cea10b407d: Deleting consensus metadata
I20260621 07:02:42.554013 21794 catalog_manager.cc:4841] TS 9ccff4ede0b0476fb433d7cea10b407d (127.2.245.194:38047): tablet 8c610b002dec4c70b08e1b90fac627bf (table test-table [id=3776ff3d0d2348a683706410c8e9879c]) successfully deleted
I20260621 07:02:42.638386 22300 tablet_replica.cc:330] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf: stopping tablet replica
I20260621 07:02:42.639034 22300 raft_consensus.cc:2229] T 8c610b002dec4c70b08e1b90fac627bf P 8a475061f65045bf99dad52082664cdf [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:42.639179 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21866
I20260621 07:02:42.658185 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21997
W20260621 07:02:42.658192 21794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.245.193:38367 (error 108)
I20260621 07:02:42.660741 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.659770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56136)
0621 07:02:42.660427 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.660546 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":356,"threads_started":1}
W20260621 07:02:42.661007 21794 catalog_manager.cc:4584] TS 8a475061f65045bf99dad52082664cdf (127.2.245.193:38367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c610b002dec4c70b08e1b90fac627bf: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
I20260621 07:02:42.665426 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.659125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.2.245.194:38047)
0621 07:02:42.659594 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.659603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.659626 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:02:42.659698 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.659702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.659804 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.659922 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:42.660379 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.661250 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.662558 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.663098 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.663216 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.664197 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:42.664206 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:42.664219 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:42.664773 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:42.664821 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:42.664866 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:42.664950 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.665086 (+ 136us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.665169 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338}
I20260621 07:02:42.666033 22119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:42.714726 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.714258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56144)
0621 07:02:42.714494 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.714583 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260621 07:02:42.715830 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.715414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56148)
0621 07:02:42.715602 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.715688 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260621 07:02:42.754258 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21997
I20260621 07:02:42.772053 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22127
I20260621 07:02:42.776978 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.776452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56152)
0621 07:02:42.776691 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.776799 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260621 07:02:42.778046 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.777631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56154)
0621 07:02:42.777818 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.777905 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 07:02:42.779277 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.772831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.2.245.195:43207)
0621 07:02:42.773082 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:42.773087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:42.773106 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:02:42.773164 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:42.773167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:42.773262 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:42.773365 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:42.773745 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.774489 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.775915 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:42.776524 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:42.776645 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:42.777890 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:42.777899 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:42.777914 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:42.778558 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:42.778604 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:42.778624 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:42.778714 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:42.778900 (+ 186us) server_negotiation.cc:299] Negotiation successful
0621 07:02:42.779046 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:02:42.780054 22249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:42.858867 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.858263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56162)
0621 07:02:42.858563 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.858678 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260621 07:02:42.860116 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:42.859700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:38367 (local address 127.0.0.1:56166)
0621 07:02:42.859908 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:42.859981 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.193:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260621 07:02:42.869462 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22127
I20260621 07:02:42.886179 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 21776
I20260621 07:02:42.887393 21837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:42.995515 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 21776
2026-06-21T07:02:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3179 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:43 dist-test-slave-9zdv krb5kdc[22312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:43 dist-test-slave-9zdv krb5kdc[22312](info): set up 2 sockets
Jun 21 07:02:43 dist-test-slave-9zdv krb5kdc[22312](info): commencing operation
krb5kdc: starting...
W20260621 07:02:44.111493 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:44 dist-test-slave-9zdv krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:02:44.362126 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46757
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:46757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:44.693431 22325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:44.693706 22325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:44.693879 22325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:44.701155 22325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:44.701264 22325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:44.701344 22325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:44.701434 22325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:44.701519 22325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:44.711506 22325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:46757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:46757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:44.712824 22325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:44.714130 22325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:44.725791 22330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:44.725749 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:44.725833 22333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:44.726639 22325 server_base.cc:1034] running on GCE node
I20260621 07:02:44.727504 22325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:44.728906 22325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:44.730108 22325 hybrid_clock.cc:648] HybridClock initialized: now 1782025364730063 us; error 36 us; skew 500 ppm
I20260621 07:02:44.730464 22325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:44 dist-test-slave-9zdv krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:44.738245 22325 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:02:44.739897 22325 webserver.cc:460] Webserver started at http://127.2.245.254:46351/ using document root <none> and password file <none>
I20260621 07:02:44.740610 22325 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:44.740742 22325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:44.741096 22325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:44.744246 22325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "fd19af6182834b50bf5b86c2b58b528f"
format_stamp: "Formatted at 2026-06-21 07:02:44 on dist-test-slave-9zdv"
server_key: "50b30f2a344971aeaae2a41917204ac1"
server_key_iv: "78f6e9d6d2791c9d63ae671713354352"
server_key_version: "encryptionkey@0"
I20260621 07:02:44.745113 22325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "fd19af6182834b50bf5b86c2b58b528f"
format_stamp: "Formatted at 2026-06-21 07:02:44 on dist-test-slave-9zdv"
server_key: "50b30f2a344971aeaae2a41917204ac1"
server_key_iv: "78f6e9d6d2791c9d63ae671713354352"
server_key_version: "encryptionkey@0"
I20260621 07:02:44.750516 22325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260621 07:02:44.754041 22340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:44.755409 22325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:44.755585 22325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "fd19af6182834b50bf5b86c2b58b528f"
format_stamp: "Formatted at 2026-06-21 07:02:44 on dist-test-slave-9zdv"
server_key: "50b30f2a344971aeaae2a41917204ac1"
server_key_iv: "78f6e9d6d2791c9d63ae671713354352"
server_key_version: "encryptionkey@0"
I20260621 07:02:44.755777 22325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:44.769605 22325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:44.773401 22325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:44.773681 22325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:44.792290 22325 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:46757
I20260621 07:02:44.792316 22391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:46757 every 8 connection(s)
I20260621 07:02:44.793655 22325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:44.796209 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22325
I20260621 07:02:44.796404 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:44.796975 3031 external_mini_cluster.cc:1428] Setting key 7a9925001e635b8480c88e333d0a60eb
I20260621 07:02:44.798676 22392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 07:02:44 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:44.813119 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: Bootstrap starting.
I20260621 07:02:44.816311 22392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:44.817309 22392 log.cc:824] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:44.820588 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: No bootstrap required, opened a new log
I20260621 07:02:44.820612 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:44.799005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.2.245.254:46757)
0621 07:02:44.799767 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:44.799779 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:44.799820 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:02:44.801390 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:44.801412 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:44.801791 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:44.802171 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:44.804105 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:44.805625 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:44.807513 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:44.808191 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:44.808322 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:44.813147 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:44.813155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:44.813170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:44.813207 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:44.816205 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:44.816992 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:44.816998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:44.817003 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:44.817166 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:44.817587 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:44.817590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:44.817592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:44.818075 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:44.818309 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:44.818897 (+ 588us) server_negotiation.cc:299] Negotiation successful
0621 07:02:44.819325 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":339,"threads_started":1}
I20260621 07:02:44.828308 22392 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } }
I20260621 07:02:44.828614 22392 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:44.828692 22392 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd19af6182834b50bf5b86c2b58b528f, State: Initialized, Role: FOLLOWER
I20260621 07:02:44.829281 22392 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } }
I20260621 07:02:44.829540 22392 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:44.829679 22392 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:44.829877 22392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:44.831923 22392 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } }
I20260621 07:02:44.832489 22392 leader_election.cc:302] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd19af6182834b50bf5b86c2b58b528f; no voters:
I20260621 07:02:44.833133 22392 leader_election.cc:288] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:44.833217 22397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:44.833900 22397 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [term 1 LEADER]: Becoming Leader. State: Replica: fd19af6182834b50bf5b86c2b58b528f, State: Running, Role: LEADER
I20260621 07:02:44.834431 22397 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } }
I20260621 07:02:44.835107 22392 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:44.837392 22398 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd19af6182834b50bf5b86c2b58b528f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } } }
I20260621 07:02:44.837496 22399 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd19af6182834b50bf5b86c2b58b528f. Latest consensus state: current_term: 1 leader_uuid: "fd19af6182834b50bf5b86c2b58b528f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd19af6182834b50bf5b86c2b58b528f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 46757 } } }
I20260621 07:02:44.837776 22398 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:44.837862 22399 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:44.838686 22405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:44.846812 22405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:44.855600 22405 catalog_manager.cc:1347] Generated new cluster ID: dee6298f38134c1782c87d3998235864
I20260621 07:02:44.855782 22405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:44.875290 22405 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:44.877182 22405 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:44.886992 22405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: Generated new TSK 0
I20260621 07:02:44.888024 22405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.245.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
Entry for principal kudu/127.2.245.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab.
I20260621 07:02:44.954747 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:45.287202 22418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:45.287505 22418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:45.287667 22418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:45.294670 22418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:45.294773 22418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:45.294886 22418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:45.305410 22418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.193.keytab
--principal=kudu/127.2.245.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:45.306640 22418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:45.308068 22418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:45.321192 22426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:45.321136 22424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:45.321784 22423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:45.323364 22418 server_base.cc:1034] running on GCE node
I20260621 07:02:45.323974 22418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:45.324949 22418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:45.326190 22418 hybrid_clock.cc:648] HybridClock initialized: now 1782025365326141 us; error 60 us; skew 500 ppm
I20260621 07:02:45.326510 22418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:45 dist-test-slave-9zdv krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025365, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:45.329603 22418 init.cc:377] Logged in from keytab as kudu/127.2.245.193@KRBTEST.COM (short username kudu)
I20260621 07:02:45.331216 22418 webserver.cc:460] Webserver started at http://127.2.245.193:43491/ using document root <none> and password file <none>
I20260621 07:02:45.331983 22418 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:45.332129 22418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:45.332448 22418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:45.335506 22418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "ee9dec5eb613486fa347c0de04f59c29"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "1827f8713757bca0106445ea63b0349a"
server_key_iv: "3b42e8f1ceed26783174febe25575482"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.336385 22418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "ee9dec5eb613486fa347c0de04f59c29"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "1827f8713757bca0106445ea63b0349a"
server_key_iv: "3b42e8f1ceed26783174febe25575482"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.341686 22418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:02:45.345126 22433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.346457 22418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:02:45.346645 22418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "ee9dec5eb613486fa347c0de04f59c29"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "1827f8713757bca0106445ea63b0349a"
server_key_iv: "3b42e8f1ceed26783174febe25575482"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.346892 22418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:45.359730 22418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:45.364091 22418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:45.364404 22418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:45.365410 22418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:45.366839 22418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:45.366992 22418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.367141 22418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:45.367213 22418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.400954 22418 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:35711
I20260621 07:02:45.401000 22545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:35711 every 8 connection(s)
I20260621 07:02:45.402282 22418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:45.409976 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22418
I20260621 07:02:45.410161 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:45.410689 3031 external_mini_cluster.cc:1428] Setting key 320dd25b1d7d968a3a4e6fc0499a1eb0
Jun 21 07:02:45 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025365, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.193@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:45.425004 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:45.406272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:53823 (local address 127.2.245.254:46757)
0621 07:02:45.406742 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:45.406747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:45.406817 (+ 70us) server_negotiation.cc:407] Connection header received
0621 07:02:45.408754 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:45.408758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:45.408836 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:45.408939 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:45.410965 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.411888 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.413785 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.414557 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.414706 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:45.418165 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:45.418174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:45.418182 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:45.418227 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:45.421593 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:45.422663 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:45.422667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:45.422669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:45.422782 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:45.423295 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:45.423298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:45.423300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:45.423554 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:45.423677 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:45.424571 (+ 894us) server_negotiation.cc:299] Negotiation successful
0621 07:02:45.424745 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":248,"threads_started":1}
I20260621 07:02:45.426587 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:45.405702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46757 (local address 127.2.245.193:53823)
0621 07:02:45.406657 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:45.406703 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:45.408473 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:45.409154 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:45.409163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:45.409833 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:45.410663 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:45.410693 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.412065 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.412071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:45.413624 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:45.413637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.414768 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.414773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:45.415063 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:45.415766 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:45.415790 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:45.417925 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:45.421757 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:45.421768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:45.421789 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:45.422537 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:45.422894 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:45.422901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:45.422908 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:45.423174 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:45.423683 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:45.423694 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:45.424166 (+ 472us) client_negotiation.cc:769] Sending connection context
0621 07:02:45.424468 (+ 302us) client_negotiation.cc:240] Negotiation successful
0621 07:02:45.425097 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":316,"threads_started":1}
I20260621 07:02:45.428212 22546 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46757
I20260621 07:02:45.428539 22546 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:45.429260 22546 heartbeater.cc:511] Master 127.2.245.254:46757 requested a full tablet report, sending...
I20260621 07:02:45.431438 22357 ts_manager.cc:194] Registered new tserver with Master: ee9dec5eb613486fa347c0de04f59c29 (127.2.245.193:35711)
I20260621 07:02:45.432874 22357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.193@KRBTEST.COM'} at 127.2.245.193:53823
WARNING: no policy specified for kudu/127.2.245.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
Entry for principal kudu/127.2.245.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab.
I20260621 07:02:45.469828 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:45.805289 22552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:45.805553 22552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:45.805716 22552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:45.813052 22552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:45.813156 22552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:45.813369 22552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:45.823014 22552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.194.keytab
--principal=kudu/127.2.245.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:45.824255 22552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:45.825531 22552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:45.839852 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:45.839956 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:45.840854 22560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:45.844678 22552 server_base.cc:1034] running on GCE node
I20260621 07:02:45.845433 22552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:45.846519 22552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:45.847752 22552 hybrid_clock.cc:648] HybridClock initialized: now 1782025365847716 us; error 44 us; skew 500 ppm
I20260621 07:02:45.848110 22552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:45 dist-test-slave-9zdv krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025365, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:45.851573 22552 init.cc:377] Logged in from keytab as kudu/127.2.245.194@KRBTEST.COM (short username kudu)
I20260621 07:02:45.853209 22552 webserver.cc:460] Webserver started at http://127.2.245.194:46821/ using document root <none> and password file <none>
I20260621 07:02:45.853962 22552 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:45.854099 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:45.854432 22552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:45.857523 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "1bb647282d3a49d98593273199c948a3"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "e5924fd149bcf2541bdc4c3164bb3adb"
server_key_iv: "f280b143eaef24e1a01b982660204755"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.858304 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "1bb647282d3a49d98593273199c948a3"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "e5924fd149bcf2541bdc4c3164bb3adb"
server_key_iv: "f280b143eaef24e1a01b982660204755"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.863780 22552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:02:45.867444 22567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.868763 22552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:02:45.868942 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "1bb647282d3a49d98593273199c948a3"
format_stamp: "Formatted at 2026-06-21 07:02:45 on dist-test-slave-9zdv"
server_key: "e5924fd149bcf2541bdc4c3164bb3adb"
server_key_iv: "f280b143eaef24e1a01b982660204755"
server_key_version: "encryptionkey@0"
I20260621 07:02:45.869146 22552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:45.887486 22552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:45.891436 22552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:45.891747 22552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:45.892686 22552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:45.894052 22552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:45.894156 22552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.894299 22552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:45.894371 22552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:45.928218 22552 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:46769
I20260621 07:02:45.928274 22679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:46769 every 8 connection(s)
I20260621 07:02:45.929563 22552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:45.934828 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22552
I20260621 07:02:45.935065 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:45.935539 3031 external_mini_cluster.cc:1428] Setting key cfb865fb6396d87e31f6661b4e9110f1
Jun 21 07:02:45 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025365, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.194@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:45.950366 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:45.932178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:51001 (local address 127.2.245.254:46757)
0621 07:02:45.932635 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:45.932640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:45.933629 (+ 989us) server_negotiation.cc:407] Connection header received
0621 07:02:45.935645 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:45.935652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:45.935753 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:45.935884 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:45.937570 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.938286 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.939975 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.940468 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.940599 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:45.944437 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:45.944442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:45.944445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:45.944476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:45.947310 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:45.948025 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:45.948037 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:45.948040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:45.948162 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:45.948624 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:45.948636 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:45.948638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:45.948907 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:45.949064 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:45.949742 (+ 678us) server_negotiation.cc:299] Negotiation successful
0621 07:02:45.949973 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260621 07:02:45.951431 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:45.932776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46757 (local address 127.2.245.194:51001)
0621 07:02:45.933464 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:45.933500 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:45.935287 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:45.936109 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:45.936117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:45.936763 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:45.937414 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:45.937434 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.938449 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.938454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:45.939809 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:45.939822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:45.940690 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:45.940699 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:45.940940 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:45.941606 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:45.941632 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:45.944194 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:45.947457 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:45.947466 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:45.947489 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:45.947892 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:45.948276 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:45.948282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:45.948286 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:45.948502 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:45.949037 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:45.949045 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:45.949387 (+ 342us) client_negotiation.cc:769] Sending connection context
0621 07:02:45.949658 (+ 271us) client_negotiation.cc:240] Negotiation successful
0621 07:02:45.950172 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":261,"threads_started":1}
I20260621 07:02:45.953023 22680 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46757
I20260621 07:02:45.953361 22680 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:45.954218 22680 heartbeater.cc:511] Master 127.2.245.254:46757 requested a full tablet report, sending...
I20260621 07:02:45.955883 22357 ts_manager.cc:194] Registered new tserver with Master: 1bb647282d3a49d98593273199c948a3 (127.2.245.194:46769)
I20260621 07:02:45.956724 22357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.194@KRBTEST.COM'} at 127.2.245.194:51001
WARNING: no policy specified for kudu/127.2.245.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
Entry for principal kudu/127.2.245.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab.
I20260621 07:02:45.993700 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:46.328991 22686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:46.329272 22686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:46.329442 22686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:46.336858 22686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:46.336997 22686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:46.337181 22686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:46.347220 22686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.195.keytab
--principal=kudu/127.2.245.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:46.348642 22686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:46.349932 22686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:46.363502 22691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:46.363601 22692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:46.363572 22694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:46.365046 22686 server_base.cc:1034] running on GCE node
I20260621 07:02:46.365700 22686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:46.366888 22686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:46.368144 22686 hybrid_clock.cc:648] HybridClock initialized: now 1782025366368089 us; error 44 us; skew 500 ppm
I20260621 07:02:46.368443 22686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:46 dist-test-slave-9zdv krb5kdc[22312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025366, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:46.371699 22686 init.cc:377] Logged in from keytab as kudu/127.2.245.195@KRBTEST.COM (short username kudu)
I20260621 07:02:46.373356 22686 webserver.cc:460] Webserver started at http://127.2.245.195:41495/ using document root <none> and password file <none>
I20260621 07:02:46.374159 22686 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:46.374292 22686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:46.374625 22686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:46.377707 22686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "bad8bd9d9a8d412491222fd9bf34ca4e"
format_stamp: "Formatted at 2026-06-21 07:02:46 on dist-test-slave-9zdv"
server_key: "2a9e37f22184582479b68795da280c1c"
server_key_iv: "2cef910b3690ae73f10db6453042215c"
server_key_version: "encryptionkey@0"
I20260621 07:02:46.378508 22686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "bad8bd9d9a8d412491222fd9bf34ca4e"
format_stamp: "Formatted at 2026-06-21 07:02:46 on dist-test-slave-9zdv"
server_key: "2a9e37f22184582479b68795da280c1c"
server_key_iv: "2cef910b3690ae73f10db6453042215c"
server_key_version: "encryptionkey@0"
I20260621 07:02:46.383862 22686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:02:46.387702 22701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:46.389066 22686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:02:46.389245 22686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "bad8bd9d9a8d412491222fd9bf34ca4e"
format_stamp: "Formatted at 2026-06-21 07:02:46 on dist-test-slave-9zdv"
server_key: "2a9e37f22184582479b68795da280c1c"
server_key_iv: "2cef910b3690ae73f10db6453042215c"
server_key_version: "encryptionkey@0"
I20260621 07:02:46.389454 22686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:46.407210 22686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:46.411360 22686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:46.411675 22686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:46.412669 22686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:46.414011 22686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:46.414083 22686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:46.414266 22686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:46.414323 22686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:46.436046 22546 heartbeater.cc:503] Master 127.2.245.254:46757 was elected leader, sending a full tablet report...
I20260621 07:02:46.448871 22686 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:46125
I20260621 07:02:46.448927 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:46125 every 8 connection(s)
I20260621 07:02:46.450237 22686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:46.458369 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22686
I20260621 07:02:46.458588 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:46.459115 3031 external_mini_cluster.cc:1428] Setting key 00b41dd80bae720e539cadbff0022636
Jun 21 07:02:46 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025366, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.195@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:02:46.472373 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.453120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:58153 (local address 127.2.245.254:46757)
0621 07:02:46.453628 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.453632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.454569 (+ 937us) server_negotiation.cc:407] Connection header received
0621 07:02:46.456505 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.456510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.456634 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.456759 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:46.458310 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.459437 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.461413 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.461951 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.462086 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.465613 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:46.465618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:46.465621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:46.465655 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:46.468820 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.469598 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.469602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.469605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.469730 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.470143 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.470147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.470149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.470462 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:46.470594 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.471596 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.471773 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":245,"threads_started":1}
I20260621 07:02:46.473425 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.453562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:46757 (local address 127.2.245.195:58153)
0621 07:02:46.454323 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.454362 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.456193 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.457018 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.457028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.457569 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:46.458129 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.458146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.459619 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.459624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.461258 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.461271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.462179 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.462186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.462427 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.463160 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:02:46.463187 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:46.465380 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:46.468978 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:46.468985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:46.468998 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:46.469439 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:46.469836 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:02:46.469840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:02:46.469842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:02:46.470019 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:02:46.470650 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:46.470656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:46.471198 (+ 542us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.471465 (+ 267us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.471917 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":329,"threads_started":1}
I20260621 07:02:46.475069 22814 heartbeater.cc:347] Connected to a master server at 127.2.245.254:46757
I20260621 07:02:46.475383 22814 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:46.476118 22814 heartbeater.cc:511] Master 127.2.245.254:46757 requested a full tablet report, sending...
I20260621 07:02:46.477653 22356 ts_manager.cc:194] Registered new tserver with Master: bad8bd9d9a8d412491222fd9bf34ca4e (127.2.245.195:46125)
I20260621 07:02:46.478624 22356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.245.195@KRBTEST.COM'} at 127.2.245.195:58153
I20260621 07:02:46.487169 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:46.504524 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.491372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.2.245.254:46757)
0621 07:02:46.491589 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.491597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.492007 (+ 410us) server_negotiation.cc:407] Connection header received
0621 07:02:46.492217 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.492221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.492287 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.492395 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:46.494064 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.494869 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.496228 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.496663 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.496797 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.499268 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:46.499272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:46.499275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:46.499309 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:46.501600 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.502310 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.502317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.502321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.502455 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.502869 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.502877 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.502880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.503286 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:46.503573 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.503929 (+ 356us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.504169 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:02:46.508471 22356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:46.515429 22356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:46.540671 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.528671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.2.245.194:46769)
0621 07:02:46.529387 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.529396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.530145 (+ 749us) server_negotiation.cc:407] Connection header received
0621 07:02:46.530270 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.530282 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.530555 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.530780 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:46.532684 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534368 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.537925 (+ 3557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.538999 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.539166 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.539435 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.540160 (+ 725us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.540352 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":439,"threads_started":1}
I20260621 07:02:46.540949 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.530135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.2.245.193:35711)
0621 07:02:46.532076 (+ 1941us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.532082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.532104 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:02:46.532183 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.532188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.532337 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.532544 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:46.533093 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534098 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.537892 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.539423 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.539569 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.539800 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.540442 (+ 642us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.540607 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1767,"thread_start_us":1636,"threads_started":1}
I20260621 07:02:46.541244 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.529788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35711 (local address 127.0.0.1:51536)
0621 07:02:46.531793 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.531812 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.531933 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.532535 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.532539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.532565 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:46.532935 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.532944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534242 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.534249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.537725 (+ 3476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.537734 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.539647 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.539653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.539801 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.539831 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.539912 (+ 81us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.540040 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1840,"mutex_wait_us":71,"thread_start_us":1663,"threads_started":1}
I20260621 07:02:46.541342 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.528255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:46769 (local address 127.0.0.1:47270)
0621 07:02:46.529852 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.529870 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.530021 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.531136 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.531144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.531181 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:46.532449 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.532469 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534505 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.534511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.537719 (+ 3208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.537734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.539199 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.539206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.539439 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.539506 (+ 67us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.539653 (+ 147us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.539870 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"mutex_wait_us":82,"spinlock_wait_cycles":9984,"thread_start_us":1309,"threads_started":1}
I20260621 07:02:46.541474 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.527808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46125 (local address 127.0.0.1:59288)
0621 07:02:46.529139 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.529182 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.529361 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.530568 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.530580 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.530622 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:46.532577 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.532588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534596 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.534602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.537176 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.537214 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.540551 (+ 3337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.540556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.540673 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.540694 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.540830 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.540973 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"mutex_wait_us":59,"thread_start_us":1132,"threads_started":1}
I20260621 07:02:46.543062 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.528042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.2.245.195:46125)
0621 07:02:46.528899 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.528909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.529557 (+ 648us) server_negotiation.cc:407] Connection header received
0621 07:02:46.529669 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.529679 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.529954 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.530199 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:46.532816 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.534382 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.537383 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.538832 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.540191 (+ 1359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.540452 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.542586 (+ 2134us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.542731 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":690,"threads_started":1}
I20260621 07:02:46.545558 22615 tablet_service.cc:1467] Processing CreateTablet for tablet 7e19dcd289804176bfbc947a8a932dcd (DEFAULT_TABLE table=test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:46.545558 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 7e19dcd289804176bfbc947a8a932dcd (DEFAULT_TABLE table=test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:46.546186 22749 tablet_service.cc:1467] Processing CreateTablet for tablet 7e19dcd289804176bfbc947a8a932dcd (DEFAULT_TABLE table=test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:46.547047 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e19dcd289804176bfbc947a8a932dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:46.547066 22615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e19dcd289804176bfbc947a8a932dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:46.547462 22749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e19dcd289804176bfbc947a8a932dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:46.560047 22833 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Bootstrap starting.
I20260621 07:02:46.562214 22834 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Bootstrap starting.
I20260621 07:02:46.564320 22833 tablet_bootstrap.cc:654] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:46.565779 22833 log.cc:824] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:46.566190 22834 tablet_bootstrap.cc:654] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:46.567796 22834 log.cc:824] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:46.568961 22835 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Bootstrap starting.
I20260621 07:02:46.573510 22835 tablet_bootstrap.cc:654] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:46.574330 22834 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: No bootstrap required, opened a new log
I20260621 07:02:46.574398 22833 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: No bootstrap required, opened a new log
I20260621 07:02:46.574632 22834 ts_tablet_manager.cc:1397] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260621 07:02:46.574756 22833 ts_tablet_manager.cc:1397] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260621 07:02:46.574926 22835 log.cc:824] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:46.578295 22835 tablet_bootstrap.cc:492] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: No bootstrap required, opened a new log
I20260621 07:02:46.578634 22835 ts_tablet_manager.cc:1397] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260621 07:02:46.582396 22834 raft_consensus.cc:348] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.582791 22834 raft_consensus.cc:374] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:46.582922 22834 raft_consensus.cc:729] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb647282d3a49d98593273199c948a3, State: Initialized, Role: FOLLOWER
I20260621 07:02:46.583647 22834 consensus_queue.cc:260] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.584748 22680 heartbeater.cc:503] Master 127.2.245.254:46757 was elected leader, sending a full tablet report...
I20260621 07:02:46.585289 22834 ts_tablet_manager.cc:1428] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
I20260621 07:02:46.585623 22833 raft_consensus.cc:348] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.586053 22833 raft_consensus.cc:374] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:46.586171 22833 raft_consensus.cc:729] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad8bd9d9a8d412491222fd9bf34ca4e, State: Initialized, Role: FOLLOWER
I20260621 07:02:46.586814 22833 consensus_queue.cc:260] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.587980 22814 heartbeater.cc:503] Master 127.2.245.254:46757 was elected leader, sending a full tablet report...
I20260621 07:02:46.588752 22833 ts_tablet_manager.cc:1428] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260621 07:02:46.591572 22835 raft_consensus.cc:348] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.592092 22835 raft_consensus.cc:374] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:46.592217 22835 raft_consensus.cc:729] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9dec5eb613486fa347c0de04f59c29, State: Initialized, Role: FOLLOWER
I20260621 07:02:46.592810 22835 consensus_queue.cc:260] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.594362 22835 ts_tablet_manager.cc:1428] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Time spent starting tablet: real 0.016s user 0.009s sys 0.004s
W20260621 07:02:46.654829 22547 tablet.cc:2369] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:46.681694 22681 tablet.cc:2369] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:46.702301 22815 tablet.cc:2369] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:46.756343 22839 raft_consensus.cc:482] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:46.756690 22839 raft_consensus.cc:504] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.758558 22839 leader_election.cc:288] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad8bd9d9a8d412491222fd9bf34ca4e (127.2.245.195:46125), ee9dec5eb613486fa347c0de04f59c29 (127.2.245.193:35711)
I20260621 07:02:46.766155 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.759139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:44867 (local address 127.2.245.195:46125)
0621 07:02:46.759391 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.759402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.759798 (+ 396us) server_negotiation.cc:407] Connection header received
0621 07:02:46.760089 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.760094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.760184 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.760284 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:46.761054 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.761992 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.764033 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.765147 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.765338 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.765413 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.765767 (+ 354us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.765885 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:02:46.766232 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.759049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:46125 (local address 127.2.245.194:44867)
0621 07:02:46.759666 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.759682 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.759859 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.760457 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.760462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.760493 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:46.760904 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.760919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.762126 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.762131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.763852 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.763864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.765371 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.765377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.765592 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.765626 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.765727 (+ 101us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.765856 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":366,"threads_started":1}
I20260621 07:02:46.766387 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.760194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:55371 (local address 127.2.245.193:35711)
0621 07:02:46.760401 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.760407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.760615 (+ 208us) server_negotiation.cc:407] Connection header received
0621 07:02:46.760746 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.760753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.760838 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.760957 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:46.761545 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.762406 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.764207 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.765172 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.765337 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.765404 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.766003 (+ 599us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.766122 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260621 07:02:46.766548 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.759865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35711 (local address 127.2.245.194:55371)
0621 07:02:46.760520 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:46.760533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:46.760643 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:46.760961 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:46.760964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:46.761005 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:46.761372 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.761385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.762543 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.762547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.764054 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:46.764068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.765561 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.765566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:46.765781 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.765819 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 07:02:46.765981 (+ 162us) client_negotiation.cc:240] Negotiation successful
0621 07:02:46.766108 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":7,"thread_start_us":351,"threads_started":1}
I20260621 07:02:46.767601 22769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e19dcd289804176bfbc947a8a932dcd" candidate_uuid: "1bb647282d3a49d98593273199c948a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" is_pre_election: true
I20260621 07:02:46.767604 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e19dcd289804176bfbc947a8a932dcd" candidate_uuid: "1bb647282d3a49d98593273199c948a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9dec5eb613486fa347c0de04f59c29" is_pre_election: true
I20260621 07:02:46.768095 22769 raft_consensus.cc:2454] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb647282d3a49d98593273199c948a3 in term 0.
I20260621 07:02:46.768107 22501 raft_consensus.cc:2454] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb647282d3a49d98593273199c948a3 in term 0.
I20260621 07:02:46.768805 22571 leader_election.cc:302] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb647282d3a49d98593273199c948a3, bad8bd9d9a8d412491222fd9bf34ca4e; no voters:
I20260621 07:02:46.769199 22839 raft_consensus.cc:2781] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:46.769361 22839 raft_consensus.cc:482] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:46.769490 22839 raft_consensus.cc:3037] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:46.771759 22839 raft_consensus.cc:504] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.772328 22839 leader_election.cc:288] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [CANDIDATE]: Term 1 election: Requested vote from peers bad8bd9d9a8d412491222fd9bf34ca4e (127.2.245.195:46125), ee9dec5eb613486fa347c0de04f59c29 (127.2.245.193:35711)
I20260621 07:02:46.772913 22769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e19dcd289804176bfbc947a8a932dcd" candidate_uuid: "1bb647282d3a49d98593273199c948a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e"
I20260621 07:02:46.772964 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e19dcd289804176bfbc947a8a932dcd" candidate_uuid: "1bb647282d3a49d98593273199c948a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9dec5eb613486fa347c0de04f59c29"
I20260621 07:02:46.773125 22769 raft_consensus.cc:3037] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:46.773216 22501 raft_consensus.cc:3037] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:46.775583 22769 raft_consensus.cc:2454] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb647282d3a49d98593273199c948a3 in term 1.
I20260621 07:02:46.775642 22501 raft_consensus.cc:2454] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb647282d3a49d98593273199c948a3 in term 1.
I20260621 07:02:46.776015 22571 leader_election.cc:302] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb647282d3a49d98593273199c948a3, bad8bd9d9a8d412491222fd9bf34ca4e; no voters:
I20260621 07:02:46.776306 22839 raft_consensus.cc:2781] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:46.776750 22839 raft_consensus.cc:686] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bb647282d3a49d98593273199c948a3, State: Running, Role: LEADER
I20260621 07:02:46.777271 22839 consensus_queue.cc:237] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } }
I20260621 07:02:46.782212 22356 catalog_manager.cc:5495] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bb647282d3a49d98593273199c948a3 (127.2.245.194). New cstate: current_term: 1 leader_uuid: "1bb647282d3a49d98593273199c948a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bb647282d3a49d98593273199c948a3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 46769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 } health_report { overall_health: UNKNOWN } } }
I20260621 07:02:46.834920 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.829721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.2.245.194:46769)
0621 07:02:46.829933 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.829937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.829955 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:02:46.830145 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.830149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.830235 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.830361 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:46.830678 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.831532 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.832842 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.833241 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.833334 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.833770 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:46.834316 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:46.834435 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.834569 (+ 134us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.834632 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:02:46.843335 22501 raft_consensus.cc:1261] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 1 FOLLOWER]: Refusing update from remote peer 1bb647282d3a49d98593273199c948a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:46.843334 22769 raft_consensus.cc:1261] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 1 FOLLOWER]: Refusing update from remote peer 1bb647282d3a49d98593273199c948a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:46.844450 22844 consensus_queue.cc:1035] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9dec5eb613486fa347c0de04f59c29" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:46.844844 22839 consensus_queue.cc:1035] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad8bd9d9a8d412491222fd9bf34ca4e" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 46125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:46.856647 22847 mvcc.cc:205] Tried to move back new op lower bound from 7299175902579617792 to 7299175902323539968. Current Snapshot: MvccSnapshot[applied={T|T < 7299175902579617792}]
I20260621 07:02:46.870926 22357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:46.871289 22357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:46.875098 22357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd19af6182834b50bf5b86c2b58b528f: Sending DeleteTablet for 3 replicas of tablet 7e19dcd289804176bfbc947a8a932dcd
I20260621 07:02:46.876292 22615 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e19dcd289804176bfbc947a8a932dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:46 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:47270
I20260621 07:02:46.876691 22481 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e19dcd289804176bfbc947a8a932dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:46 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:51536
I20260621 07:02:46.877055 22749 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e19dcd289804176bfbc947a8a932dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:46 UTC) from {username='kudu', principal='kudu/127.2.245.254@KRBTEST.COM'} at 127.0.0.1:59288
I20260621 07:02:46.877534 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22418
I20260621 07:02:46.887735 22857 tablet_replica.cc:330] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: stopping tablet replica
I20260621 07:02:46.889236 22857 raft_consensus.cc:2229] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:46.891909 22857 raft_consensus.cc:2258] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:46.894164 22859 tablet_replica.cc:330] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: stopping tablet replica
I20260621 07:02:46.894475 22858 tablet_replica.cc:330] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: stopping tablet replica
I20260621 07:02:46.895018 22858 raft_consensus.cc:2229] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:46.895615 22858 raft_consensus.cc:2258] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:46.895632 22857 ts_tablet_manager.cc:1905] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:46.896456 22859 raft_consensus.cc:2229] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:46.897022 22859 raft_consensus.cc:2258] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:46.899199 22858 ts_tablet_manager.cc:1905] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:46.901110 22859 ts_tablet_manager.cc:1905] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 21 07:02:46 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.193@KRBTEST.COM
I20260621 07:02:46.903491 22857 ts_tablet_manager.cc:1918] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:46.903776 22857 log.cc:1196] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/7e19dcd289804176bfbc947a8a932dcd
I20260621 07:02:46.904412 22857 ts_tablet_manager.cc:1939] T 7e19dcd289804176bfbc947a8a932dcd P 1bb647282d3a49d98593273199c948a3: Deleting consensus metadata
I20260621 07:02:46.906527 22341 catalog_manager.cc:4841] TS 1bb647282d3a49d98593273199c948a3 (127.2.245.194:46769): tablet 7e19dcd289804176bfbc947a8a932dcd (table test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]) successfully deleted
I20260621 07:02:46.908077 22858 ts_tablet_manager.cc:1918] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:46.908257 22858 log.cc:1196] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/7e19dcd289804176bfbc947a8a932dcd
I20260621 07:02:46.908860 22858 ts_tablet_manager.cc:1939] T 7e19dcd289804176bfbc947a8a932dcd P ee9dec5eb613486fa347c0de04f59c29: Deleting consensus metadata
I20260621 07:02:46.910718 22344 catalog_manager.cc:4841] TS ee9dec5eb613486fa347c0de04f59c29 (127.2.245.193:35711): tablet 7e19dcd289804176bfbc947a8a932dcd (table test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]) successfully deleted
I20260621 07:02:46.911616 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:46.878799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51540 (local address 127.2.245.193:35711)
0621 07:02:46.879010 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:46.879016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:46.889230 (+ 10214us) server_negotiation.cc:407] Connection header received
0621 07:02:46.889704 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:46.889711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:46.889799 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:46.889875 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:46.892210 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.893420 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.896388 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:46.897035 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:46.897153 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:46.902963 (+ 5810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:46.902981 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:46.903000 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:46.903060 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:46.907485 (+ 4425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.908552 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.908565 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.908572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.908760 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:46.909239 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:46.909244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:46.909246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:46.909722 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:46.910056 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:46.910383 (+ 327us) server_negotiation.cc:299] Negotiation successful
0621 07:02:46.911173 (+ 790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:02:46.912575 22541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:46.917313 22859 ts_tablet_manager.cc:1918] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:46.917510 22859 log.cc:1196] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/7e19dcd289804176bfbc947a8a932dcd
I20260621 07:02:46.917930 22859 ts_tablet_manager.cc:1939] T 7e19dcd289804176bfbc947a8a932dcd P bad8bd9d9a8d412491222fd9bf34ca4e: Deleting consensus metadata
I20260621 07:02:46.919589 22344 catalog_manager.cc:4841] TS bad8bd9d9a8d412491222fd9bf34ca4e (127.2.245.195:46125): tablet 7e19dcd289804176bfbc947a8a932dcd (table test-table [id=eb1d5f2b05ae40d2be3f46dc6ad7a6c9]) successfully deleted
I20260621 07:02:47.024037 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22418
I20260621 07:02:47.043727 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22552
Jun 21 07:02:47 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.194@KRBTEST.COM
I20260621 07:02:47.062220 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:47.044506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.2.245.194:46769)
0621 07:02:47.044675 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:47.044679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:47.044755 (+ 76us) server_negotiation.cc:407] Connection header received
0621 07:02:47.044946 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:47.044950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:47.045069 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:47.045157 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:47.046836 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.047768 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.049674 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.050077 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.050182 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:47.055643 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:47.055657 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:47.055673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:47.055723 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:47.059104 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:47.059937 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:47.059947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:47.059951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:47.060142 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:47.060577 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:47.060582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:47.060586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:47.061051 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:47.061399 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:47.061554 (+ 155us) server_negotiation.cc:299] Negotiation successful
0621 07:02:47.061798 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:02:47.063036 22675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:47.179518 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22552
I20260621 07:02:47.199049 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22686
Jun 21 07:02:47 dist-test-slave-9zdv krb5kdc[22312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.195@KRBTEST.COM
I20260621 07:02:47.214560 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:47.199749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.2.245.195:46125)
0621 07:02:47.200014 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:47.200020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:47.200040 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:02:47.200219 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:47.200225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:47.200303 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:47.200428 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:47.201722 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.202492 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.204035 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.204441 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.204581 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:47.209022 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:47.209036 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:47.209053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:47.209089 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:47.211816 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:47.212490 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:47.212505 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:47.212508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:47.212643 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:47.213019 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:47.213025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:47.213028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:47.213381 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:47.213684 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:47.213983 (+ 299us) server_negotiation.cc:299] Negotiation successful
0621 07:02:47.214200 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:02:47.215250 22809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:47.304332 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22686
I20260621 07:02:47.321188 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22325
I20260621 07:02:47.322285 22387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:47.434064 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22325
2026-06-21T07:02:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4439 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-21T07:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:47Z Disabled control of system clock
I20260621 07:02:47.512871 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:37595
--webserver_interface=127.2.245.254
--webserver_port=0
--builtin_ntp_servers=127.2.245.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:47.846773 22868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:47.847172 22868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:47.847322 22868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:47.854579 22868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:47.854683 22868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:47.854775 22868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:47.854884 22868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:47.855000 22868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:47.864158 22868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:37595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:47.865458 22868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:47.867293 22868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:47.879243 22874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:47.879295 22876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:47.879330 22873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:47.880049 22868 server_base.cc:1034] running on GCE node
I20260621 07:02:47.880720 22868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:47.882054 22868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:47.883249 22868 hybrid_clock.cc:648] HybridClock initialized: now 1782025367883193 us; error 59 us; skew 500 ppm
I20260621 07:02:47.883615 22868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:47.885824 22868 webserver.cc:460] Webserver started at http://127.2.245.254:36739/ using document root <none> and password file <none>
I20260621 07:02:47.886508 22868 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:47.886620 22868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:47.887014 22868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:47.890105 22868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "3551aa9296de4978b61fcf9b79f8a60f"
format_stamp: "Formatted at 2026-06-21 07:02:47 on dist-test-slave-9zdv"
server_key: "37b08cefb682ee3de0f6a10648d2d923"
server_key_iv: "c4f6ba2e864bd685754225783068d856"
server_key_version: "encryptionkey@0"
I20260621 07:02:47.890923 22868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "3551aa9296de4978b61fcf9b79f8a60f"
format_stamp: "Formatted at 2026-06-21 07:02:47 on dist-test-slave-9zdv"
server_key: "37b08cefb682ee3de0f6a10648d2d923"
server_key_iv: "c4f6ba2e864bd685754225783068d856"
server_key_version: "encryptionkey@0"
I20260621 07:02:47.895952 22868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 07:02:47.899307 22882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:47.900665 22868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 07:02:47.900853 22868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "3551aa9296de4978b61fcf9b79f8a60f"
format_stamp: "Formatted at 2026-06-21 07:02:47 on dist-test-slave-9zdv"
server_key: "37b08cefb682ee3de0f6a10648d2d923"
server_key_iv: "c4f6ba2e864bd685754225783068d856"
server_key_version: "encryptionkey@0"
I20260621 07:02:47.901072 22868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:47.936695 22868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:47.937636 22868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:47.937954 22868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:47.957320 22868 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:37595
I20260621 07:02:47.957327 22933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:37595 every 8 connection(s)
I20260621 07:02:47.958719 22868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:47.962450 22934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:47.965548 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22868
I20260621 07:02:47.965734 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:47.966327 3031 external_mini_cluster.cc:1428] Setting key 1d9aa6c59ca8c417cadc8b2c62f8f309
I20260621 07:02:47.974532 22934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: Bootstrap starting.
I20260621 07:02:47.977826 22934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:47.978915 22934 log.cc:824] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:47.980458 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:47.968396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.2.245.254:37595)
0621 07:02:47.969356 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:47.969368 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:47.969408 (+ 40us) server_negotiation.cc:407] Connection header received
0621 07:02:47.971223 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:47.971247 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:47.971784 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:47.972171 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:47.972773 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.973997 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.975512 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:47.976273 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:47.976526 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:47.977673 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:47.977681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:47.977694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:47.978331 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:47.978427 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:47.978449 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:47.978576 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:47.978819 (+ 243us) server_negotiation.cc:299] Negotiation successful
0621 07:02:47.979198 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":322,"threads_started":1}
I20260621 07:02:47.982079 22934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: No bootstrap required, opened a new log
I20260621 07:02:47.989845 22934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } }
I20260621 07:02:47.990192 22934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:47.990269 22934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3551aa9296de4978b61fcf9b79f8a60f, State: Initialized, Role: FOLLOWER
I20260621 07:02:47.990842 22934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } }
I20260621 07:02:47.991145 22934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:47.991253 22934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:47.991367 22934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:47.993456 22934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } }
I20260621 07:02:47.993911 22934 leader_election.cc:302] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3551aa9296de4978b61fcf9b79f8a60f; no voters:
I20260621 07:02:47.994478 22934 leader_election.cc:288] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:47.994632 22939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:47.995432 22939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [term 1 LEADER]: Becoming Leader. State: Replica: 3551aa9296de4978b61fcf9b79f8a60f, State: Running, Role: LEADER
I20260621 07:02:47.995967 22939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } }
I20260621 07:02:47.996953 22934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:47.999361 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3551aa9296de4978b61fcf9b79f8a60f. Latest consensus state: current_term: 1 leader_uuid: "3551aa9296de4978b61fcf9b79f8a60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } } }
I20260621 07:02:47.999384 22940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3551aa9296de4978b61fcf9b79f8a60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3551aa9296de4978b61fcf9b79f8a60f" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 37595 } } }
I20260621 07:02:47.999867 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:47.999876 22940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:48.001375 22945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:48.008313 22945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:48.020092 22945 catalog_manager.cc:1347] Generated new cluster ID: 94f51d67f1c84a88aa18a3ea2e56e4b9
I20260621 07:02:48.020246 22945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:48.038960 22945 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:48.040227 22945 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:48.056067 22945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: Generated new TSK 0
I20260621 07:02:48.057178 22945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:48.068066 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.193:0
--local_ip_for_outbound_sockets=127.2.245.193
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--builtin_ntp_servers=127.2.245.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 07:02:48.404512 22958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:48.404786 22958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:48.404940 22958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:48.412386 22958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:48.412492 22958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:48.412647 22958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.193
I20260621 07:02:48.421713 22958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.245.193
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.193
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:48.423044 22958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:48.424847 22958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:48.438228 22963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:48.438143 22964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:48.438190 22966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:48.439204 22958 server_base.cc:1034] running on GCE node
I20260621 07:02:48.439900 22958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:48.440874 22958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:48.442114 22958 hybrid_clock.cc:648] HybridClock initialized: now 1782025368442072 us; error 60 us; skew 500 ppm
I20260621 07:02:48.442449 22958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:48.444860 22958 webserver.cc:460] Webserver started at http://127.2.245.193:43089/ using document root <none> and password file <none>
I20260621 07:02:48.445562 22958 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:48.445694 22958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:48.446008 22958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:48.448959 22958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data/instance:
uuid: "8791b9570aea4e058a7d18560041a712"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "d846f9fe590714a80d64496f007471a4"
server_key_iv: "eaea0d03b2c0d0289d0aa108bf134106"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.449728 22958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance:
uuid: "8791b9570aea4e058a7d18560041a712"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "d846f9fe590714a80d64496f007471a4"
server_key_iv: "eaea0d03b2c0d0289d0aa108bf134106"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.454598 22958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260621 07:02:48.458176 22972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.459687 22958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:48.459867 22958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal
uuid: "8791b9570aea4e058a7d18560041a712"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "d846f9fe590714a80d64496f007471a4"
server_key_iv: "eaea0d03b2c0d0289d0aa108bf134106"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.460094 22958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:48.493978 22958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:48.494823 22958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:48.495196 22958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:48.496232 22958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:48.497769 22958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:48.497892 22958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.498029 22958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:48.498117 22958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.531272 22958 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.193:35357
I20260621 07:02:48.531354 23084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.193:35357 every 8 connection(s)
I20260621 07:02:48.532629 22958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/data/info.pb
I20260621 07:02:48.536753 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 22958
I20260621 07:02:48.536934 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal/instance
I20260621 07:02:48.537424 3031 external_mini_cluster.cc:1428] Setting key f26cd3d4732d3e82274e63452a5e5b8e
I20260621 07:02:48.540223 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.194:0
--local_ip_for_outbound_sockets=127.2.245.194
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--builtin_ntp_servers=127.2.245.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:48.546798 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:48.535169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:33643 (local address 127.2.245.254:37595)
0621 07:02:48.536386 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:48.536391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:48.537392 (+ 1001us) server_negotiation.cc:407] Connection header received
0621 07:02:48.539422 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:48.539429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:48.539519 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:48.539631 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:48.540753 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.541971 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.543616 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.544343 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.544661 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:48.545816 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:48.545825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:48.545833 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:48.545861 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:48.545915 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:48.545932 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:48.546030 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:48.546234 (+ 204us) server_negotiation.cc:299] Negotiation successful
0621 07:02:48.546346 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":1018,"threads_started":1}
I20260621 07:02:48.547612 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:48.536038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37595 (local address 127.2.245.193:33643)
0621 07:02:48.537218 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:48.537264 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:48.539178 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:48.539824 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:48.539833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:48.539941 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:48.540584 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:48.540603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.542120 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.542125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:48.543456 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:48.543474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.544559 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.544564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:48.544764 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:48.545497 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:48.545499 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:48.545540 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:48.545620 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:48.546034 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:48.546041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:48.546047 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 07:02:48.546188 (+ 141us) client_negotiation.cc:240] Negotiation successful
0621 07:02:48.546487 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":381,"threads_started":1}
I20260621 07:02:48.549386 23085 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37595
I20260621 07:02:48.549719 23085 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:48.550465 23085 heartbeater.cc:511] Master 127.2.245.254:37595 requested a full tablet report, sending...
I20260621 07:02:48.552817 22899 ts_manager.cc:194] Registered new tserver with Master: 8791b9570aea4e058a7d18560041a712 (127.2.245.193:35357)
I20260621 07:02:48.555104 22899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.193:33643
W20260621 07:02:48.870391 23089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:48.870661 23089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:48.870743 23089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:48.877934 23089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:48.878023 23089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:48.878125 23089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.194
I20260621 07:02:48.887014 23089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.245.194
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.194
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:48.888213 23089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:48.889866 23089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:48.902988 23094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:48.902981 23095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:48.903051 23097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:48.903864 23089 server_base.cc:1034] running on GCE node
I20260621 07:02:48.904433 23089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:48.905385 23089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:48.906649 23089 hybrid_clock.cc:648] HybridClock initialized: now 1782025368906611 us; error 63 us; skew 500 ppm
I20260621 07:02:48.907050 23089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:48.909286 23089 webserver.cc:460] Webserver started at http://127.2.245.194:39831/ using document root <none> and password file <none>
I20260621 07:02:48.909991 23089 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:48.910120 23089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:48.910429 23089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:48.913367 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data/instance:
uuid: "c6e829cb31a14c7b9d813f236fd32cc3"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "ddda02dbc6f99f14ad1a508b60423436"
server_key_iv: "9331db32bdeeb0b0ada024072533606b"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.914145 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance:
uuid: "c6e829cb31a14c7b9d813f236fd32cc3"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "ddda02dbc6f99f14ad1a508b60423436"
server_key_iv: "9331db32bdeeb0b0ada024072533606b"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.919051 23089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:02:48.922276 23103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.923591 23089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260621 07:02:48.923759 23089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal
uuid: "c6e829cb31a14c7b9d813f236fd32cc3"
format_stamp: "Formatted at 2026-06-21 07:02:48 on dist-test-slave-9zdv"
server_key: "ddda02dbc6f99f14ad1a508b60423436"
server_key_iv: "9331db32bdeeb0b0ada024072533606b"
server_key_version: "encryptionkey@0"
I20260621 07:02:48.923972 23089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:48.940670 23089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:48.941447 23089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:48.941744 23089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:48.942698 23089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:48.944151 23089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:48.944278 23089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.944412 23089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:48.944509 23089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:48.982784 23089 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.194:41743
I20260621 07:02:48.982833 23215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.194:41743 every 8 connection(s)
I20260621 07:02:48.984099 23089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/data/info.pb
I20260621 07:02:48.987872 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23089
I20260621 07:02:48.988047 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal/instance
I20260621 07:02:48.988549 3031 external_mini_cluster.cc:1428] Setting key f7f028f1ecd3b53e87307aa14a681e1c
I20260621 07:02:48.991283 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.245.195:0
--local_ip_for_outbound_sockets=127.2.245.195
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--builtin_ntp_servers=127.2.245.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 07:02:48.996619 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:48.986578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.194:42605 (local address 127.2.245.254:37595)
0621 07:02:48.986790 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:48.986797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:48.988308 (+ 1511us) server_negotiation.cc:407] Connection header received
0621 07:02:48.990094 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:48.990100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:48.990191 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:48.990305 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:48.991515 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.992231 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.993846 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.994251 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.994392 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:48.995773 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:48.995783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:48.995786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:48.995805 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:48.995835 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:48.995847 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:48.995946 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:48.996198 (+ 252us) server_negotiation.cc:299] Negotiation successful
0621 07:02:48.996297 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:02:48.997670 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:48.987149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37595 (local address 127.2.245.194:42605)
0621 07:02:48.988156 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:48.988200 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:48.989833 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:48.990536 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:48.990544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:48.990630 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:48.991290 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:48.991319 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.992394 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.992398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:48.993699 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:48.993711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:48.994474 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:48.994481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:48.994713 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:48.995456 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:48.995459 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:48.995499 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:48.995574 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:48.995945 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:48.995957 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:48.995967 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 07:02:48.996170 (+ 203us) client_negotiation.cc:240] Negotiation successful
0621 07:02:48.996561 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":318,"threads_started":1}
I20260621 07:02:48.999251 23216 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37595
I20260621 07:02:48.999593 23216 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:49.000524 23216 heartbeater.cc:511] Master 127.2.245.254:37595 requested a full tablet report, sending...
I20260621 07:02:49.002120 22899 ts_manager.cc:194] Registered new tserver with Master: c6e829cb31a14c7b9d813f236fd32cc3 (127.2.245.194:41743)
I20260621 07:02:49.003466 22899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.194:42605
W20260621 07:02:49.321800 23219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:49.322137 23219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:49.322225 23219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:49.329555 23219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:49.329658 23219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:49.329761 23219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.245.195
I20260621 07:02:49.338904 23219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.245.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.245.195
--webserver_port=0
--tserver_master_addrs=127.2.245.254:37595
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.245.195
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:49.340215 23219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:49.342000 23219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:49.355334 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:49.355370 23227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:49.355415 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:49.356243 23219 server_base.cc:1034] running on GCE node
I20260621 07:02:49.356838 23219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:49.357858 23219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:49.359083 23219 hybrid_clock.cc:648] HybridClock initialized: now 1782025369359007 us; error 65 us; skew 500 ppm
I20260621 07:02:49.359400 23219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260621 07:02:49.361624 23219 webserver.cc:460] Webserver started at http://127.2.245.195:46487/ using document root <none> and password file <none>
I20260621 07:02:49.362339 23219 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:49.362473 23219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:49.362843 23219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:49.365914 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data/instance:
uuid: "801459de4be1435e98bf2fd2bf29dfc3"
format_stamp: "Formatted at 2026-06-21 07:02:49 on dist-test-slave-9zdv"
server_key: "d3dd7b524a11753b6dadd651771eed87"
server_key_iv: "30ea24dc19e77bf34eb97a1d91f16665"
server_key_version: "encryptionkey@0"
I20260621 07:02:49.366725 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance:
uuid: "801459de4be1435e98bf2fd2bf29dfc3"
format_stamp: "Formatted at 2026-06-21 07:02:49 on dist-test-slave-9zdv"
server_key: "d3dd7b524a11753b6dadd651771eed87"
server_key_iv: "30ea24dc19e77bf34eb97a1d91f16665"
server_key_version: "encryptionkey@0"
I20260621 07:02:49.371908 23219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260621 07:02:49.375358 23233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:49.376639 23219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260621 07:02:49.376829 23219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal
uuid: "801459de4be1435e98bf2fd2bf29dfc3"
format_stamp: "Formatted at 2026-06-21 07:02:49 on dist-test-slave-9zdv"
server_key: "d3dd7b524a11753b6dadd651771eed87"
server_key_iv: "30ea24dc19e77bf34eb97a1d91f16665"
server_key_version: "encryptionkey@0"
I20260621 07:02:49.377056 23219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:49.395546 23219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:49.396369 23219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:49.396683 23219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:49.397747 23219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:02:49.399369 23219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:02:49.399489 23219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:49.399627 23219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:02:49.399714 23219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:49.433696 23219 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.195:38631
I20260621 07:02:49.433743 23345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.195:38631 every 8 connection(s)
I20260621 07:02:49.435199 23219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/data/info.pb
I20260621 07:02:49.439173 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23219
I20260621 07:02:49.439354 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal/instance
I20260621 07:02:49.439916 3031 external_mini_cluster.cc:1428] Setting key f9f75178603b5f114787fc7b5d34c7ad
I20260621 07:02:49.448026 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.437764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.195:52713 (local address 127.2.245.254:37595)
0621 07:02:49.437955 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.437960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.439564 (+ 1604us) server_negotiation.cc:407] Connection header received
0621 07:02:49.441734 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.441739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.441824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.441917 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:49.443279 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.444027 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.445512 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.445933 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.446037 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.447369 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:49.447374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:49.447377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:49.447395 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:49.447425 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:49.447437 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:49.447518 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.447717 (+ 199us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.447791 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:02:49.448990 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.438270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:37595 (local address 127.2.245.195:52713)
0621 07:02:49.439397 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.439446 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.441477 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.442195 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.442207 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.442329 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:02:49.443127 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.443147 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.444162 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.444165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.445366 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.445376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.446175 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.446180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.446386 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.447074 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:02:49.447076 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:02:49.447113 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:02:49.447190 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:02:49.447529 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:02:49.447537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:02:49.447542 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.447670 (+ 128us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.447946 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":265,"threads_started":1}
I20260621 07:02:49.450356 23346 heartbeater.cc:347] Connected to a master server at 127.2.245.254:37595
I20260621 07:02:49.450685 23346 heartbeater.cc:464] Registering TS with master...
I20260621 07:02:49.451527 23346 heartbeater.cc:511] Master 127.2.245.254:37595 requested a full tablet report, sending...
I20260621 07:02:49.452966 22899 ts_manager.cc:194] Registered new tserver with Master: 801459de4be1435e98bf2fd2bf29dfc3 (127.2.245.195:38631)
I20260621 07:02:49.453796 22899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.245.195:52713
I20260621 07:02:49.455217 3031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:02:49.465358 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.459231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.2.245.254:37595)
0621 07:02:49.459415 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.459431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.459869 (+ 438us) server_negotiation.cc:407] Connection header received
0621 07:02:49.460093 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.460102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.460174 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.460282 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:49.460795 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.461485 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.462877 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.463365 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.463517 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.464640 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:49.464645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:49.464648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:49.464665 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:49.464696 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:49.464708 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:49.464782 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.464907 (+ 125us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.465012 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:02:49.468880 22899 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:02:49.475692 22899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:02:49.511601 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.487920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.2.245.193:35357)
0621 07:02:49.502904 (+ 14984us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.502913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.502996 (+ 83us) server_negotiation.cc:407] Connection header received
0621 07:02:49.503093 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.503100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.503352 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.503559 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:49.504351 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.506002 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.508863 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.510260 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.510396 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.510658 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.511125 (+ 467us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.511275 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14837,"thread_start_us":14725,"threads_started":1}
I20260621 07:02:49.511588 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.490338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.2.245.194:41743)
0621 07:02:49.501378 (+ 11040us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.501390 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.502457 (+ 1067us) server_negotiation.cc:407] Connection header received
0621 07:02:49.502576 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.502584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.502851 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.503083 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:49.504603 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.506369 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.508512 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.509924 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.510074 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.510323 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.511024 (+ 701us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.511237 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10896,"thread_start_us":10772,"threads_started":1}
I20260621 07:02:49.511620 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.487672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.193:35357 (local address 127.0.0.1:53306)
0621 07:02:49.500000 (+ 12328us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.500051 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.500273 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.503594 (+ 3321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.503599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.503624 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:49.504086 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.504098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.506161 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.506166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.508651 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.508665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.510516 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.510536 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.510727 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.510787 (+ 60us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.510906 (+ 119us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.511174 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11962,"mutex_wait_us":166,"thread_start_us":14233,"threads_started":1}
I20260621 07:02:49.511788 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.488068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:38631 (local address 127.0.0.1:41282)
0621 07:02:49.500733 (+ 12665us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.500759 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.500905 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.503089 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.503097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.503129 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:49.503601 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.503651 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.505723 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.505732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.508448 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.508464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.511004 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.511011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.511226 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.511254 (+ 28us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.511335 (+ 81us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.511445 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12497,"mutex_wait_us":144,"thread_start_us":11380,"threads_started":1}
I20260621 07:02:49.512504 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.488563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.2.245.195:38631)
0621 07:02:49.502583 (+ 14020us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.502591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.502620 (+ 29us) server_negotiation.cc:407] Connection header received
0621 07:02:49.502696 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.502702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.502866 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.503083 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:49.503853 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.505586 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.509262 (+ 3676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.510460 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.510600 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.510863 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.512024 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.512228 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13835,"thread_start_us":13670,"threads_started":1}
I20260621 07:02:49.512544 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.489957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:41743 (local address 127.0.0.1:38350)
0621 07:02:49.502169 (+ 12212us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.502190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.502319 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.503334 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.503338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.503369 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:49.503997 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.504034 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.506503 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.506516 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.508303 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.508320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.510471 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.510478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.510722 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.510779 (+ 57us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.510921 (+ 142us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.512315 (+ 1394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12081,"mutex_wait_us":1356,"thread_start_us":10432,"threads_started":1}
I20260621 07:02:49.516121 23020 tablet_service.cc:1467] Processing CreateTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 (DEFAULT_TABLE table=test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:49.517333 23020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d3f887b96240d39b8e29d20d305ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:49.516960 23281 tablet_service.cc:1467] Processing CreateTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 (DEFAULT_TABLE table=test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:49.518196 23281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d3f887b96240d39b8e29d20d305ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:49.520048 23151 tablet_service.cc:1467] Processing CreateTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 (DEFAULT_TABLE table=test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:02:49.521286 23151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d3f887b96240d39b8e29d20d305ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:49.528847 23364 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Bootstrap starting.
I20260621 07:02:49.532927 23364 tablet_bootstrap.cc:654] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:49.534221 23364 log.cc:824] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:49.538159 23367 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Bootstrap starting.
I20260621 07:02:49.540710 23365 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Bootstrap starting.
I20260621 07:02:49.544605 23367 tablet_bootstrap.cc:654] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:49.545149 23365 tablet_bootstrap.cc:654] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:49.545941 23367 log.cc:824] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:49.548225 23364 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: No bootstrap required, opened a new log
I20260621 07:02:49.548413 23365 log.cc:824] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:49.548511 23364 ts_tablet_manager.cc:1397] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.008s
I20260621 07:02:49.549633 23367 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: No bootstrap required, opened a new log
I20260621 07:02:49.549947 23367 ts_tablet_manager.cc:1397] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260621 07:02:49.557374 23365 tablet_bootstrap.cc:492] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: No bootstrap required, opened a new log
I20260621 07:02:49.557701 23365 ts_tablet_manager.cc:1397] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260621 07:02:49.558203 23085 heartbeater.cc:503] Master 127.2.245.254:37595 was elected leader, sending a full tablet report...
I20260621 07:02:49.559286 23364 raft_consensus.cc:348] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.559779 23364 raft_consensus.cc:374] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:49.559870 23364 raft_consensus.cc:729] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8791b9570aea4e058a7d18560041a712, State: Initialized, Role: FOLLOWER
I20260621 07:02:49.560479 23364 consensus_queue.cc:260] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.560870 23367 raft_consensus.cc:348] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.561399 23367 raft_consensus.cc:374] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:49.561527 23367 raft_consensus.cc:729] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e829cb31a14c7b9d813f236fd32cc3, State: Initialized, Role: FOLLOWER
I20260621 07:02:49.562379 23364 ts_tablet_manager.cc:1428] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260621 07:02:49.562291 23367 consensus_queue.cc:260] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.563539 23216 heartbeater.cc:503] Master 127.2.245.254:37595 was elected leader, sending a full tablet report...
I20260621 07:02:49.565415 23367 ts_tablet_manager.cc:1428] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260621 07:02:49.569802 23365 raft_consensus.cc:348] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.570183 23365 raft_consensus.cc:374] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:49.570289 23365 raft_consensus.cc:729] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801459de4be1435e98bf2fd2bf29dfc3, State: Initialized, Role: FOLLOWER
I20260621 07:02:49.570833 23365 consensus_queue.cc:260] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.571868 23346 heartbeater.cc:503] Master 127.2.245.254:37595 was elected leader, sending a full tablet report...
I20260621 07:02:49.572510 23365 ts_tablet_manager.cc:1428] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260621 07:02:49.687007 23347 tablet.cc:2369] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:02:49.736145 23217 tablet.cc:2369] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:49.748288 23370 raft_consensus.cc:482] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:02:49.748636 23370 raft_consensus.cc:504] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.750348 23370 leader_election.cc:288] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801459de4be1435e98bf2fd2bf29dfc3 (127.2.245.195:38631), c6e829cb31a14c7b9d813f236fd32cc3 (127.2.245.194:41743)
I20260621 07:02:49.757839 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.750983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:37569 (local address 127.2.245.195:38631)
0621 07:02:49.751211 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.751218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.752221 (+ 1003us) server_negotiation.cc:407] Connection header received
0621 07:02:49.752419 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.752422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.752479 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.752600 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:49.753251 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.753985 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.756005 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.756868 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.756990 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.757078 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.757448 (+ 370us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.757589 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260621 07:02:49.757903 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.751012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.245.193:44379 (local address 127.2.245.194:41743)
0621 07:02:49.751214 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.751222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.752117 (+ 895us) server_negotiation.cc:407] Connection header received
0621 07:02:49.752331 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.752338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.752441 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.752516 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:02:49.753191 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.753894 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.756071 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.756995 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.757141 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.757210 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.757512 (+ 302us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.757640 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:02:49.758021 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.750927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.194:41743 (local address 127.2.245.193:44379)
0621 07:02:49.751941 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.751957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.752134 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.752573 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.752580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.752606 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:49.753020 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.753032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.754030 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.754036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.755817 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.755829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.757207 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.757212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.757365 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.757387 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.757484 (+ 97us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.757640 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":114,"thread_start_us":487,"threads_started":1}
I20260621 07:02:49.758064 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.751088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.195:38631 (local address 127.2.245.193:37569)
0621 07:02:49.752017 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0621 07:02:49.752101 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0621 07:02:49.752285 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:02:49.752633 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:02:49.752641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:02:49.752678 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:02:49.753072 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.753081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.754136 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.754140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.755814 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:02:49.755828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.757072 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.757081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:02:49.757283 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.757307 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:02:49.757415 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:02:49.757622 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":115,"thread_start_us":575,"threads_started":1}
I20260621 07:02:49.759142 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d3f887b96240d39b8e29d20d305ae6" candidate_uuid: "8791b9570aea4e058a7d18560041a712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801459de4be1435e98bf2fd2bf29dfc3" is_pre_election: true
I20260621 07:02:49.759145 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d3f887b96240d39b8e29d20d305ae6" candidate_uuid: "8791b9570aea4e058a7d18560041a712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" is_pre_election: true
I20260621 07:02:49.759632 23301 raft_consensus.cc:2454] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8791b9570aea4e058a7d18560041a712 in term 0.
I20260621 07:02:49.759631 23171 raft_consensus.cc:2454] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8791b9570aea4e058a7d18560041a712 in term 0.
I20260621 07:02:49.760309 22974 leader_election.cc:302] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 801459de4be1435e98bf2fd2bf29dfc3, 8791b9570aea4e058a7d18560041a712, c6e829cb31a14c7b9d813f236fd32cc3; no voters:
I20260621 07:02:49.760730 23370 raft_consensus.cc:2781] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:02:49.760880 23370 raft_consensus.cc:482] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:02:49.760972 23370 raft_consensus.cc:3037] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:49.763214 23370 raft_consensus.cc:504] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.763895 23370 leader_election.cc:288] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [CANDIDATE]: Term 1 election: Requested vote from peers 801459de4be1435e98bf2fd2bf29dfc3 (127.2.245.195:38631), c6e829cb31a14c7b9d813f236fd32cc3 (127.2.245.194:41743)
I20260621 07:02:49.764240 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d3f887b96240d39b8e29d20d305ae6" candidate_uuid: "8791b9570aea4e058a7d18560041a712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801459de4be1435e98bf2fd2bf29dfc3"
I20260621 07:02:49.764436 23301 raft_consensus.cc:3037] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:49.764358 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d3f887b96240d39b8e29d20d305ae6" candidate_uuid: "8791b9570aea4e058a7d18560041a712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e829cb31a14c7b9d813f236fd32cc3"
I20260621 07:02:49.764603 23171 raft_consensus.cc:3037] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:49.766664 23301 raft_consensus.cc:2454] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8791b9570aea4e058a7d18560041a712 in term 1.
I20260621 07:02:49.766857 23171 raft_consensus.cc:2454] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8791b9570aea4e058a7d18560041a712 in term 1.
I20260621 07:02:49.767118 22976 leader_election.cc:302] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801459de4be1435e98bf2fd2bf29dfc3, 8791b9570aea4e058a7d18560041a712; no voters:
I20260621 07:02:49.767400 23370 raft_consensus.cc:2781] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:49.768042 23370 raft_consensus.cc:686] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 1 LEADER]: Becoming Leader. State: Replica: 8791b9570aea4e058a7d18560041a712, State: Running, Role: LEADER
I20260621 07:02:49.768555 23370 consensus_queue.cc:237] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } }
I20260621 07:02:49.773319 22898 catalog_manager.cc:5495] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8791b9570aea4e058a7d18560041a712 (127.2.245.193). New cstate: current_term: 1 leader_uuid: "8791b9570aea4e058a7d18560041a712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8791b9570aea4e058a7d18560041a712" member_type: VOTER last_known_addr { host: "127.2.245.193" port: 35357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 } health_report { overall_health: UNKNOWN } } }
W20260621 07:02:49.784945 23086 tablet.cc:2369] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:02:49.805554 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.799973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53322 (local address 127.2.245.193:35357)
0621 07:02:49.800181 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.800189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.800231 (+ 42us) server_negotiation.cc:407] Connection header received
0621 07:02:49.800503 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.800506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.800590 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.800755 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:02:49.801151 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.801848 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.803349 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.803754 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.803897 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.804379 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:02:49.804866 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:02:49.805036 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.805086 (+ 50us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.805161 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 07:02:49.814060 23171 raft_consensus.cc:1261] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 1 FOLLOWER]: Refusing update from remote peer 8791b9570aea4e058a7d18560041a712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:49.814090 23301 raft_consensus.cc:1261] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 1 FOLLOWER]: Refusing update from remote peer 8791b9570aea4e058a7d18560041a712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:02:49.815083 23375 consensus_queue.cc:1035] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801459de4be1435e98bf2fd2bf29dfc3" member_type: VOTER last_known_addr { host: "127.2.245.195" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:49.815402 23370 consensus_queue.cc:1035] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6e829cb31a14c7b9d813f236fd32cc3" member_type: VOTER last_known_addr { host: "127.2.245.194" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:02:49.829263 23380 mvcc.cc:205] Tried to move back new op lower bound from 7299175914747826176 to 7299175914575437824. Current Snapshot: MvccSnapshot[applied={T|T < 7299175914747826176}]
I20260621 07:02:49.841993 22899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:49.842284 22899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:02:49.846102 22899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3551aa9296de4978b61fcf9b79f8a60f: Sending DeleteTablet for 3 replicas of tablet 55d3f887b96240d39b8e29d20d305ae6
I20260621 07:02:49.847250 23020 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:49 UTC) from {username='slave'} at 127.0.0.1:53306
I20260621 07:02:49.847283 23151 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:49 UTC) from {username='slave'} at 127.0.0.1:38350
I20260621 07:02:49.847283 23281 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d3f887b96240d39b8e29d20d305ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:02:49 UTC) from {username='slave'} at 127.0.0.1:41282
I20260621 07:02:49.852675 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22958
I20260621 07:02:49.855492 23389 tablet_replica.cc:330] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: stopping tablet replica
I20260621 07:02:49.856086 23389 raft_consensus.cc:2229] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:49.856752 23389 raft_consensus.cc:2258] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:49.859617 23388 tablet_replica.cc:330] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: stopping tablet replica
I20260621 07:02:49.860101 23388 raft_consensus.cc:2229] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:02:49.860867 23388 raft_consensus.cc:2258] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:49.862457 23390 tablet_replica.cc:330] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: stopping tablet replica
I20260621 07:02:49.864147 23388 ts_tablet_manager.cc:1905] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:49.864374 23389 ts_tablet_manager.cc:1905] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:49.865514 23390 raft_consensus.cc:2229] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:02:49.866043 23390 raft_consensus.cc:2258] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:02:49.868219 23390 ts_tablet_manager.cc:1905] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:02:49.871626 23389 ts_tablet_manager.cc:1918] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:49.871840 23389 log.cc:1196] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-1/wal/wals/55d3f887b96240d39b8e29d20d305ae6
I20260621 07:02:49.872463 23389 ts_tablet_manager.cc:1939] T 55d3f887b96240d39b8e29d20d305ae6 P c6e829cb31a14c7b9d813f236fd32cc3: Deleting consensus metadata
I20260621 07:02:49.874423 22884 catalog_manager.cc:4841] TS c6e829cb31a14c7b9d813f236fd32cc3 (127.2.245.194:41743): tablet 55d3f887b96240d39b8e29d20d305ae6 (table test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]) successfully deleted
I20260621 07:02:49.876999 23388 ts_tablet_manager.cc:1918] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:49.877256 23388 log.cc:1196] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-0/wal/wals/55d3f887b96240d39b8e29d20d305ae6
I20260621 07:02:49.877875 23388 ts_tablet_manager.cc:1939] T 55d3f887b96240d39b8e29d20d305ae6 P 8791b9570aea4e058a7d18560041a712: Deleting consensus metadata
I20260621 07:02:49.878477 23390 ts_tablet_manager.cc:1918] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:02:49.878620 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:49.855532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.2.245.193:35357)
0621 07:02:49.856068 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:49.856077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:49.865250 (+ 9173us) server_negotiation.cc:407] Connection header received
0621 07:02:49.866660 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:49.866666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:49.866761 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:49.866835 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:49.867887 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.869115 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.871068 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:49.871692 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:49.871787 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:49.876764 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:49.876776 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:49.876794 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:49.877645 (+ 851us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:49.877706 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:49.877732 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:49.877827 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:49.878089 (+ 262us) server_negotiation.cc:299] Negotiation successful
0621 07:02:49.878198 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374}
I20260621 07:02:49.878985 23390 log.cc:1196] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Deleting WAL directory at /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782025057166075-3031-0/minicluster-data/ts-2/wal/wals/55d3f887b96240d39b8e29d20d305ae6
I20260621 07:02:49.879693 23080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:49.879700 22883 catalog_manager.cc:4841] TS 8791b9570aea4e058a7d18560041a712 (127.2.245.193:35357): tablet 55d3f887b96240d39b8e29d20d305ae6 (table test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]) successfully deleted
I20260621 07:02:49.879834 23390 ts_tablet_manager.cc:1939] T 55d3f887b96240d39b8e29d20d305ae6 P 801459de4be1435e98bf2fd2bf29dfc3: Deleting consensus metadata
I20260621 07:02:49.881520 22886 catalog_manager.cc:4841] TS 801459de4be1435e98bf2fd2bf29dfc3 (127.2.245.195:38631): tablet 55d3f887b96240d39b8e29d20d305ae6 (table test-table [id=0b8ce332a4464ec48fddac0969f4c4b7]) successfully deleted
I20260621 07:02:49.979737 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22958
I20260621 07:02:49.999409 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23089
I20260621 07:02:50.006639 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:50.000352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.2.245.194:41743)
0621 07:02:50.000527 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:50.000531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:50.000549 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:02:50.000688 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:50.000692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:50.000779 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:50.000886 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:50.001327 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:50.002129 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:50.003499 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:50.003959 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:50.004065 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:50.005146 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:50.005155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:50.005170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:50.005888 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:50.005936 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:50.005958 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:50.006057 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:50.006192 (+ 135us) server_negotiation.cc:299] Negotiation successful
0621 07:02:50.006283 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:02:50.007566 23211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:50.096239 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23089
I20260621 07:02:50.113951 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23219
I20260621 07:02:50.120409 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:50.114741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.2.245.195:38631)
0621 07:02:50.114912 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:50.114916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:50.114988 (+ 72us) server_negotiation.cc:407] Connection header received
0621 07:02:50.115087 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:50.115091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:50.115190 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:50.115277 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:50.115655 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:50.116399 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:50.117619 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:50.118011 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:50.118104 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:50.119223 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:50.119233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:50.119247 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:50.119786 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:50.119831 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:50.119850 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:50.119943 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:50.120076 (+ 133us) server_negotiation.cc:299] Negotiation successful
0621 07:02:50.120161 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 07:02:50.121286 23341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:50.209277 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23219
I20260621 07:02:50.226337 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 22868
I20260621 07:02:50.227540 22929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:50.336746 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 22868
2026-06-21T07:02:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2901 ms)
[----------] 6 tests from EncryptionPolicyTest (22089 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:50 dist-test-slave-9zdv krb5kdc[23398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:50 dist-test-slave-9zdv krb5kdc[23398](info): set up 2 sockets
Jun 21 07:02:50 dist-test-slave-9zdv krb5kdc[23398](info): commencing operation
krb5kdc: starting...
W20260621 07:02:51.455628 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:51 dist-test-slave-9zdv krb5kdc[23398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:02:51.715610 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:52.050830 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:52.051187 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:52.051354 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:52.059252 23411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:52.059392 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:52.059471 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:52.059571 23411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:52.059626 23411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:52.069653 23411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:52.070998 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:52.072415 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:52.084342 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:52.084407 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:52.084453 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:52.085675 23411 server_base.cc:1034] running on GCE node
I20260621 07:02:52.086370 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:52.087780 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:52.088975 23411 hybrid_clock.cc:648] HybridClock initialized: now 1782025372088932 us; error 35 us; skew 500 ppm
I20260621 07:02:52.089316 23411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:52 dist-test-slave-9zdv krb5kdc[23398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025372, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:52.092702 23411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:02:52.094134 23411 webserver.cc:460] Webserver started at http://127.0.0.1:46215/ using document root <none> and password file <none>
I20260621 07:02:52.094779 23411 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:52.094894 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:52.095206 23411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:52.098165 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "a9db195dd93848a39e5d746946d221db"
format_stamp: "Formatted at 2026-06-21 07:02:52 on dist-test-slave-9zdv"
server_key: "fca82ada5240708e26a55f6e6735f885"
server_key_iv: "260904e724ba978344f7feb55e0535f2"
server_key_version: "encryptionkey@0"
I20260621 07:02:52.099105 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "a9db195dd93848a39e5d746946d221db"
format_stamp: "Formatted at 2026-06-21 07:02:52 on dist-test-slave-9zdv"
server_key: "fca82ada5240708e26a55f6e6735f885"
server_key_iv: "260904e724ba978344f7feb55e0535f2"
server_key_version: "encryptionkey@0"
I20260621 07:02:52.104202 23411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:02:52.107615 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:52.108940 23411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:02:52.109109 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "a9db195dd93848a39e5d746946d221db"
format_stamp: "Formatted at 2026-06-21 07:02:52 on dist-test-slave-9zdv"
server_key: "fca82ada5240708e26a55f6e6735f885"
server_key_iv: "260904e724ba978344f7feb55e0535f2"
server_key_version: "encryptionkey@0"
I20260621 07:02:52.109310 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:52.133001 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:52.136881 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:52.137179 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:52.156409 23411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36647
I20260621 07:02:52.156446 23477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36647 every 8 connection(s)
I20260621 07:02:52.157831 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:52.161569 23478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:52.161826 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23411
I20260621 07:02:52.162082 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:52.162747 3031 external_mini_cluster.cc:1428] Setting key d68200f0786a5aa40c8f75444d1fd2af
I20260621 07:02:52.176127 23478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db: Bootstrap starting.
Jun 21 07:02:52 dist-test-slave-9zdv krb5kdc[23398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 07:02:52.179461 23478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:52.180879 23478 log.cc:824] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:52.184273 23478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db: No bootstrap required, opened a new log
I20260621 07:02:52.188123 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:52.164855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33729 (local address 127.0.0.1:36647)
0621 07:02:52.165660 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:52.165675 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:52.165724 (+ 49us) server_negotiation.cc:407] Connection header received
0621 07:02:52.168361 (+ 2637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:52.168391 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:52.168946 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:52.169410 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:52.170777 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:52.172785 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:52.174157 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:52.175269 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:52.175469 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:52.179479 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:52.179492 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:52.179513 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:52.179563 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:52.183462 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:52.184194 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:52.184202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:52.184210 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:52.184443 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:52.184868 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:52.184873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:52.184876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:52.185541 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:52.185853 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:52.186277 (+ 424us) server_negotiation.cc:299] Negotiation successful
0621 07:02:52.186640 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":315,"threads_started":1}
I20260621 07:02:52.192575 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:02:52.194188 23478 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } }
I20260621 07:02:52.194550 23478 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:52.194674 23478 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9db195dd93848a39e5d746946d221db, State: Initialized, Role: FOLLOWER
I20260621 07:02:52.195300 23478 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } }
I20260621 07:02:52.195538 23478 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:52.195674 23478 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:52.195847 23478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:52.198487 23478 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } }
I20260621 07:02:52.199201 23478 leader_election.cc:302] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9db195dd93848a39e5d746946d221db; no voters:
I20260621 07:02:52.199863 23478 leader_election.cc:288] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:52.199959 23488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:52.201558 23488 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [term 1 LEADER]: Becoming Leader. State: Replica: a9db195dd93848a39e5d746946d221db, State: Running, Role: LEADER
I20260621 07:02:52.202265 23478 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:52.202128 23488 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } }
I20260621 07:02:52.208115 23489 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9db195dd93848a39e5d746946d221db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } } }
I20260621 07:02:52.208499 23489 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:52.209116 23490 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9db195dd93848a39e5d746946d221db. Latest consensus state: current_term: 1 leader_uuid: "a9db195dd93848a39e5d746946d221db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9db195dd93848a39e5d746946d221db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36647 } } }
I20260621 07:02:52.209556 23490 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:52.218852 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:52.197343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59371 (local address 127.0.0.1:36647)
0621 07:02:52.197577 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:52.197585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:52.198117 (+ 532us) server_negotiation.cc:407] Connection header received
0621 07:02:52.198294 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:52.198297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:52.198355 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:52.198601 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:52.200492 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:52.201625 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:52.203477 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:52.204447 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:52.204612 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:52.208694 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:52.208705 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:52.208713 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:52.208770 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:52.213396 (+ 4626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:52.214808 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:52.214818 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:52.214824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:52.215070 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:52.215760 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:52.215768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:52.215774 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:52.216272 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:52.216648 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:52.218102 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0621 07:02:52.218343 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:02:52.220542 23493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:52.227413 23493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:52.237589 23493 catalog_manager.cc:1347] Generated new cluster ID: 1143b8d726444f5387d775141b48e1ac
I20260621 07:02:52.237711 23493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:52.249406 23493 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:52.250777 23493 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:52.257452 23493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9db195dd93848a39e5d746946d221db: Generated new TSK 0
I20260621 07:02:52.258248 23493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:52.268832 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23411
I20260621 07:02:52.270380 23473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:52.361212 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23411
W20260621 07:02:52.378379 23396 connection.cc:535] client connection to 127.0.0.1:36647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36647): Connection reset by peer (error 104)
2026-06-21T07:02:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:52 dist-test-slave-9zdv krb5kdc[23511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:52 dist-test-slave-9zdv krb5kdc[23511](info): set up 2 sockets
Jun 21 07:02:52 dist-test-slave-9zdv krb5kdc[23511](info): commencing operation
krb5kdc: starting...
W20260621 07:02:53.463650 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:53 dist-test-slave-9zdv krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:02:53.722149 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:54.056057 23524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:54.056398 23524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:54.056571 23524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:54.063864 23524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:54.064007 23524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:54.064116 23524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:54.064230 23524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:54.064307 23524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:54.074149 23524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:54.075551 23524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:54.076874 23524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:54.089118 23532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:54.089056 23529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:54.089133 23530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:54.089908 23524 server_base.cc:1034] running on GCE node
I20260621 07:02:54.090637 23524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:54.092101 23524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:54.093307 23524 hybrid_clock.cc:648] HybridClock initialized: now 1782025374093273 us; error 32 us; skew 500 ppm
I20260621 07:02:54.093621 23524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:54 dist-test-slave-9zdv krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:54.096843 23524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:02:54.098310 23524 webserver.cc:460] Webserver started at http://127.0.0.1:42511/ using document root <none> and password file <none>
I20260621 07:02:54.099089 23524 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:54.099229 23524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:54.099545 23524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:54.102561 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "33f2eeb36baf4e6a9212cd29df969820"
format_stamp: "Formatted at 2026-06-21 07:02:54 on dist-test-slave-9zdv"
server_key: "fa86384dc11ab8d39057e4e5a4f2d113"
server_key_iv: "86d14ebcde46fefc6f2e8c834f7b437f"
server_key_version: "encryptionkey@0"
I20260621 07:02:54.103462 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "33f2eeb36baf4e6a9212cd29df969820"
format_stamp: "Formatted at 2026-06-21 07:02:54 on dist-test-slave-9zdv"
server_key: "fa86384dc11ab8d39057e4e5a4f2d113"
server_key_iv: "86d14ebcde46fefc6f2e8c834f7b437f"
server_key_version: "encryptionkey@0"
I20260621 07:02:54.108738 23524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:02:54.112262 23539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:54.113639 23524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:02:54.113843 23524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "33f2eeb36baf4e6a9212cd29df969820"
format_stamp: "Formatted at 2026-06-21 07:02:54 on dist-test-slave-9zdv"
server_key: "fa86384dc11ab8d39057e4e5a4f2d113"
server_key_iv: "86d14ebcde46fefc6f2e8c834f7b437f"
server_key_version: "encryptionkey@0"
I20260621 07:02:54.114049 23524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:54.139012 23524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:54.143375 23524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:54.143694 23524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:54.163159 23524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43579
I20260621 07:02:54.163213 23590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43579 every 8 connection(s)
I20260621 07:02:54.164593 23524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:54.166982 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23524
I20260621 07:02:54.167186 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:54.167649 3031 external_mini_cluster.cc:1428] Setting key d0ac1267eb3092f9ba7dcecf8ed8fb39
I20260621 07:02:54.168882 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:54.182744 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:54.169527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34427 (local address 127.0.0.1:43579)
0621 07:02:54.170363 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:54.170378 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:54.170431 (+ 53us) server_negotiation.cc:407] Connection header received
0621 07:02:54.172328 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:54.172356 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:54.172880 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:54.173335 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:54.173844 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:54.175953 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:54.177326 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:54.178362 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:54.178546 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:54.179745 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:54.179754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:54.179777 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:54.180571 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:54.180673 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:54.180710 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:54.180877 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:54.181191 (+ 314us) server_negotiation.cc:299] Negotiation successful
0621 07:02:54.181372 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":346,"threads_started":1}
I20260621 07:02:54.183928 23591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820: Bootstrap starting.
I20260621 07:02:54.186643 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:02:54.187265 23591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:54.188476 23591 log.cc:824] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:54.191907 23591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820: No bootstrap required, opened a new log
I20260621 07:02:54.198819 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:54.191705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52961 (local address 127.0.0.1:43579)
0621 07:02:54.191943 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:54.191952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:54.192419 (+ 467us) server_negotiation.cc:407] Connection header received
0621 07:02:54.192603 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:54.192610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:54.192668 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:54.192747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:54.193361 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:54.194149 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:54.196166 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:54.196638 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:54.196733 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:54.198000 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:54.198005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:54.198008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:54.198026 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:54.198073 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:54.198091 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:54.198175 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:54.198385 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 07:02:54.198429 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:02:54.201674 23591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } }
I20260621 07:02:54.202117 23591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:54.202224 23591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f2eeb36baf4e6a9212cd29df969820, State: Initialized, Role: FOLLOWER
I20260621 07:02:54.202813 23591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } }
I20260621 07:02:54.203187 23591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:54.203342 23591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:54.203527 23591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:54.205653 23591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } }
I20260621 07:02:54.206187 23591 leader_election.cc:302] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33f2eeb36baf4e6a9212cd29df969820; no voters:
I20260621 07:02:54.206843 23591 leader_election.cc:288] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:54.206997 23601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:54.207624 23601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [term 1 LEADER]: Becoming Leader. State: Replica: 33f2eeb36baf4e6a9212cd29df969820, State: Running, Role: LEADER
I20260621 07:02:54.208294 23601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } }
I20260621 07:02:54.209214 23591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:54.211510 23603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33f2eeb36baf4e6a9212cd29df969820. Latest consensus state: current_term: 1 leader_uuid: "33f2eeb36baf4e6a9212cd29df969820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } } }
I20260621 07:02:54.212060 23603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:54.211494 23602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33f2eeb36baf4e6a9212cd29df969820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f2eeb36baf4e6a9212cd29df969820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43579 } } }
I20260621 07:02:54.212563 23602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:54.214242 23608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:54.219945 23608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:54.229643 23608 catalog_manager.cc:1347] Generated new cluster ID: 44643044a73b4d6dbc91802acc8274b4
I20260621 07:02:54.229779 23608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:54.243094 23608 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:54.244449 23608 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:54.255133 23608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33f2eeb36baf4e6a9212cd29df969820: Generated new TSK 0
I20260621 07:02:54.255980 23608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:54.263652 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23524
W20260621 07:02:54.265304 23586 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34427
W20260621 07:02:54.266040 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23524: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:02:54.266247 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23524
W20260621 07:02:54.283479 23509 connection.cc:535] client connection to 127.0.0.1:43579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43579): Connection reset by peer (error 104)
2026-06-21T07:02:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1904 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:54 dist-test-slave-9zdv krb5kdc[23622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:54 dist-test-slave-9zdv krb5kdc[23622](info): set up 2 sockets
Jun 21 07:02:54 dist-test-slave-9zdv krb5kdc[23622](info): commencing operation
krb5kdc: starting...
W20260621 07:02:55.371587 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:55 dist-test-slave-9zdv krb5kdc[23622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:02:55.639302 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:55.976627 23635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:55.976933 23635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:55.977123 23635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:55.984486 23635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:55.984599 23635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:55.984707 23635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:55.984824 23635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:55.984907 23635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:55.994648 23635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38301
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:55.995997 23635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:55.997359 23635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:56.009624 23643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:56.009721 23640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:56.009691 23641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:56.010612 23635 server_base.cc:1034] running on GCE node
I20260621 07:02:56.011512 23635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:56.013029 23635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:56.014273 23635 hybrid_clock.cc:648] HybridClock initialized: now 1782025376014222 us; error 46 us; skew 500 ppm
I20260621 07:02:56.014647 23635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:56 dist-test-slave-9zdv krb5kdc[23622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:56.023427 23635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:02:56.025198 23635 webserver.cc:460] Webserver started at http://127.0.0.1:41787/ using document root <none> and password file <none>
I20260621 07:02:56.025954 23635 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:56.026062 23635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:56.026403 23635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:56.030085 23635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "372c6ddd92074e6fada6fee176e82a47"
format_stamp: "Formatted at 2026-06-21 07:02:56 on dist-test-slave-9zdv"
server_key: "8ff1ba981e1a5cab28a168e5663fd37f"
server_key_iv: "0fe1d353c1c81bd7e226a261d778b555"
server_key_version: "encryptionkey@0"
I20260621 07:02:56.031275 23635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "372c6ddd92074e6fada6fee176e82a47"
format_stamp: "Formatted at 2026-06-21 07:02:56 on dist-test-slave-9zdv"
server_key: "8ff1ba981e1a5cab28a168e5663fd37f"
server_key_iv: "0fe1d353c1c81bd7e226a261d778b555"
server_key_version: "encryptionkey@0"
I20260621 07:02:56.036618 23635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260621 07:02:56.040217 23650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:56.041564 23635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:02:56.041777 23635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "372c6ddd92074e6fada6fee176e82a47"
format_stamp: "Formatted at 2026-06-21 07:02:56 on dist-test-slave-9zdv"
server_key: "8ff1ba981e1a5cab28a168e5663fd37f"
server_key_iv: "0fe1d353c1c81bd7e226a261d778b555"
server_key_version: "encryptionkey@0"
I20260621 07:02:56.041955 23635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:56.047972 23635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:56.052382 23635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:56.052706 23635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:56.071848 23635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38301
I20260621 07:02:56.071900 23701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38301 every 8 connection(s)
I20260621 07:02:56.073312 23635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:56.074290 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23635
I20260621 07:02:56.074472 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:56.075059 3031 external_mini_cluster.cc:1428] Setting key a5db90b234307681028b42cf4c15f955
I20260621 07:02:56.078033 23702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:56.082895 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:56.076960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49217 (local address 127.0.0.1:38301)
0621 07:02:56.077735 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:56.077746 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:56.077786 (+ 40us) server_negotiation.cc:407] Connection header received
0621 07:02:56.079557 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:56.079579 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:56.079890 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:56.080285 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:56.081326 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:56.081331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:56.081345 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:56.081361 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:56.081423 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:56.081451 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:56.081681 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:56.081810 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 07:02:56.081980 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":323,"threads_started":1}
I20260621 07:02:56.086649 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:02:56.089576 23702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47: Bootstrap starting.
I20260621 07:02:56.093184 23702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:56.093432 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:56.090970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58145 (local address 127.0.0.1:38301)
0621 07:02:56.091160 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:56.091168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:56.091640 (+ 472us) server_negotiation.cc:407] Connection header received
0621 07:02:56.091756 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:56.091761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:56.091819 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:56.091926 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:56.092771 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:56.092777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:56.092780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:02:56.092798 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:56.092842 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:02:56.092863 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:56.092985 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:56.093042 (+ 57us) server_negotiation.cc:299] Negotiation successful
0621 07:02:56.093105 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:02:56.094362 23702 log.cc:824] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:56.097095 23702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47: No bootstrap required, opened a new log
I20260621 07:02:56.104791 23702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260621 07:02:56.105217 23702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:56.105314 23702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372c6ddd92074e6fada6fee176e82a47, State: Initialized, Role: FOLLOWER
I20260621 07:02:56.105867 23702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260621 07:02:56.106093 23702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:56.106259 23702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:56.106426 23702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:56.108857 23702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260621 07:02:56.109359 23702 leader_election.cc:302] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 372c6ddd92074e6fada6fee176e82a47; no voters:
I20260621 07:02:56.109939 23702 leader_election.cc:288] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:56.110080 23712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:56.110810 23712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [term 1 LEADER]: Becoming Leader. State: Replica: 372c6ddd92074e6fada6fee176e82a47, State: Running, Role: LEADER
I20260621 07:02:56.111351 23712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } }
I20260621 07:02:56.112053 23702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:56.114235 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "372c6ddd92074e6fada6fee176e82a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } } }
I20260621 07:02:56.114393 23714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 372c6ddd92074e6fada6fee176e82a47. Latest consensus state: current_term: 1 leader_uuid: "372c6ddd92074e6fada6fee176e82a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372c6ddd92074e6fada6fee176e82a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38301 } } }
I20260621 07:02:56.114707 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:56.114732 23714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47 [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:56.118759 23721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:56.123021 23721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:56.131192 23721 catalog_manager.cc:1347] Generated new cluster ID: 82463fbdf50f4734b980e01d0051ee7b
I20260621 07:02:56.131310 23721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:56.148712 23721 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:56.149230 23721 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:56.163758 23721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 372c6ddd92074e6fada6fee176e82a47: Generated new TSK 0
I20260621 07:02:56.164947 23721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:56.176843 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23635
W20260621 07:02:56.178068 23697 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49217
W20260621 07:02:56.179102 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23635: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:02:56.179276 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23635
2026-06-21T07:02:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:56 dist-test-slave-9zdv krb5kdc[23733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:56 dist-test-slave-9zdv krb5kdc[23733](info): set up 2 sockets
Jun 21 07:02:56 dist-test-slave-9zdv krb5kdc[23733](info): commencing operation
krb5kdc: starting...
W20260621 07:02:57.287654 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:57 dist-test-slave-9zdv krb5kdc[23733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:02:57.550743 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:57.888912 23746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:57.889293 23746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:57.889461 23746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:57.896878 23746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:57.897048 23746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:57.897161 23746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:57.897289 23746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:57.897370 23746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:57.907557 23746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:57.908915 23746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:57.910251 23746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:57.922508 23752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:57.922565 23754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:57.922539 23751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:57.923168 23746 server_base.cc:1034] running on GCE node
I20260621 07:02:57.924103 23746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:57.925529 23746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:57.926800 23746 hybrid_clock.cc:648] HybridClock initialized: now 1782025377926753 us; error 63 us; skew 500 ppm
I20260621 07:02:57.927225 23746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:57 dist-test-slave-9zdv krb5kdc[23733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:57.930496 23746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:02:57.931969 23746 webserver.cc:460] Webserver started at http://127.0.0.1:36535/ using document root <none> and password file <none>
I20260621 07:02:57.932617 23746 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:57.932729 23746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:57.932968 23746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:57.936009 23746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "35604757d234489f9089b4f9e0d071fa"
format_stamp: "Formatted at 2026-06-21 07:02:57 on dist-test-slave-9zdv"
server_key: "aebd4eabb17fae1526fb748dac2d50a1"
server_key_iv: "6b6b8d63a6f90c74ea4e4d7e124488be"
server_key_version: "encryptionkey@0"
I20260621 07:02:57.936717 23746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "35604757d234489f9089b4f9e0d071fa"
format_stamp: "Formatted at 2026-06-21 07:02:57 on dist-test-slave-9zdv"
server_key: "aebd4eabb17fae1526fb748dac2d50a1"
server_key_iv: "6b6b8d63a6f90c74ea4e4d7e124488be"
server_key_version: "encryptionkey@0"
I20260621 07:02:57.942430 23746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260621 07:02:57.946007 23761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:57.947486 23746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:02:57.947642 23746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "35604757d234489f9089b4f9e0d071fa"
format_stamp: "Formatted at 2026-06-21 07:02:57 on dist-test-slave-9zdv"
server_key: "aebd4eabb17fae1526fb748dac2d50a1"
server_key_iv: "6b6b8d63a6f90c74ea4e4d7e124488be"
server_key_version: "encryptionkey@0"
I20260621 07:02:57.947844 23746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:57.975665 23746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:57.980291 23746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:57.980535 23746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:02:57.999410 23746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38937
I20260621 07:02:57.999450 23812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38937 every 8 connection(s)
I20260621 07:02:58.000831 23746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:02:58.005049 23813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:02:58.006484 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23746
I20260621 07:02:58.006675 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:02:58.007189 3031 external_mini_cluster.cc:1428] Setting key 849764819b55843f0cd15ea786077a8b
I20260621 07:02:58.018112 23813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa: Bootstrap starting.
Jun 21 07:02:58 dist-test-slave-9zdv krb5kdc[23733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 07:02:58.021664 23813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa: Neither blocks nor log segments found. Creating new log.
I20260621 07:02:58.022735 23813 log.cc:824] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa: Log is configured to *not* fsync() on all Append() calls
I20260621 07:02:58.025938 23813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa: No bootstrap required, opened a new log
I20260621 07:02:58.030560 23816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:58.009082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49709 (local address 127.0.0.1:38937)
0621 07:02:58.009905 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:58.009918 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:58.009959 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:02:58.011644 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:58.011668 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:58.012192 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:58.012626 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:58.014082 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:58.015604 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:58.016989 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:58.017666 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:58.017879 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:58.022494 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:58.022503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:58.022517 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:58.022554 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:58.026033 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:58.027098 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:58.027105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:58.027111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:58.027297 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:58.027756 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:58.027759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:58.027761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:58.028321 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:58.028538 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:58.029036 (+ 498us) server_negotiation.cc:299] Negotiation successful
0621 07:02:58.029463 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":389,"threads_started":1}
I20260621 07:02:58.034718 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:02:58.035916 23813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } }
I20260621 07:02:58.036352 23813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:02:58.036451 23813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35604757d234489f9089b4f9e0d071fa, State: Initialized, Role: FOLLOWER
I20260621 07:02:58.037110 23813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } }
I20260621 07:02:58.037406 23813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:02:58.037549 23813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:02:58.037721 23813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:02:58.040867 23813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } }
I20260621 07:02:58.041426 23813 leader_election.cc:302] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35604757d234489f9089b4f9e0d071fa; no voters:
I20260621 07:02:58.042078 23813 leader_election.cc:288] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:02:58.042309 23823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:02:58.042984 23823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [term 1 LEADER]: Becoming Leader. State: Replica: 35604757d234489f9089b4f9e0d071fa, State: Running, Role: LEADER
I20260621 07:02:58.043582 23823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } }
I20260621 07:02:58.045465 23823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35604757d234489f9089b4f9e0d071fa. Latest consensus state: current_term: 1 leader_uuid: "35604757d234489f9089b4f9e0d071fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } } }
I20260621 07:02:58.045729 23823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:58.046833 23825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35604757d234489f9089b4f9e0d071fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35604757d234489f9089b4f9e0d071fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38937 } } }
I20260621 07:02:58.047216 23825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [sys.catalog]: This master's current role is: LEADER
I20260621 07:02:58.049753 23813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:02:58.050007 23827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:02:58.055902 23827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:02:58.060473 23816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:02:58.039991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41717 (local address 127.0.0.1:38937)
0621 07:02:58.040190 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0621 07:02:58.040197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:02:58.040588 (+ 391us) server_negotiation.cc:407] Connection header received
0621 07:02:58.040734 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:02:58.040741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:02:58.040823 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:02:58.040958 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:02:58.042907 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:58.044056 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:58.045815 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:02:58.046548 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:02:58.046682 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:02:58.050179 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:02:58.050185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:02:58.050189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:02:58.050224 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:02:58.055181 (+ 4957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:58.056379 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:58.056387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:58.056393 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:58.056640 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:02:58.057230 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:02:58.057237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:02:58.057243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:02:58.057740 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:02:58.059395 (+ 1655us) server_negotiation.cc:1035] Waiting for connection context
0621 07:02:58.059605 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 07:02:58.059929 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:02:58.067919 23827 catalog_manager.cc:1347] Generated new cluster ID: e6c0eea1d7cd405594465bbd822fa5c4
I20260621 07:02:58.068068 23827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:02:58.089789 23827 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:02:58.091245 23827 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:02:58.098295 23827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35604757d234489f9089b4f9e0d071fa: Generated new TSK 0
I20260621 07:02:58.099417 23827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:02:58.112497 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23746
I20260621 07:02:58.113684 23808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:02:58.210187 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23746
2026-06-21T07:02:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:02:58 dist-test-slave-9zdv krb5kdc[23845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:02:58 dist-test-slave-9zdv krb5kdc[23845](info): set up 2 sockets
Jun 21 07:02:58 dist-test-slave-9zdv krb5kdc[23845](info): commencing operation
krb5kdc: starting...
W20260621 07:02:59.315807 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:02:59 dist-test-slave-9zdv krb5kdc[23845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:02:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:02:59.580338 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:02:59.919070 23858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:02:59.919371 23858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:02:59.919575 23858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:02:59.927313 23858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:02:59.927424 23858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:02:59.927553 23858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:02:59.927634 23858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:02:59.927735 23858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:02:59.937973 23858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:02:59.939442 23858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:02:59.940779 23858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:02:59.952997 23863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:59.952929 23866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:02:59.952927 23864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:02:59.953892 23858 server_base.cc:1034] running on GCE node
I20260621 07:02:59.954802 23858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:02:59.956310 23858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:02:59.957578 23858 hybrid_clock.cc:648] HybridClock initialized: now 1782025379957544 us; error 58 us; skew 500 ppm
I20260621 07:02:59.957945 23858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:02:59 dist-test-slave-9zdv krb5kdc[23845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:02:59.961247 23858 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:02:59.962745 23858 webserver.cc:460] Webserver started at http://127.0.0.1:41761/ using document root <none> and password file <none>
I20260621 07:02:59.963531 23858 fs_manager.cc:359] Metadata directory not provided
I20260621 07:02:59.963719 23858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:02:59.964067 23858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:02:59.967198 23858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "c73db7149435481b803fdc4ea6d47e78"
format_stamp: "Formatted at 2026-06-21 07:02:59 on dist-test-slave-9zdv"
server_key: "c7322cb35a90bf095e2491d6abe2addd"
server_key_iv: "c30c01a757ad4e7c92335cb352029d0d"
server_key_version: "encryptionkey@0"
I20260621 07:02:59.968010 23858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "c73db7149435481b803fdc4ea6d47e78"
format_stamp: "Formatted at 2026-06-21 07:02:59 on dist-test-slave-9zdv"
server_key: "c7322cb35a90bf095e2491d6abe2addd"
server_key_iv: "c30c01a757ad4e7c92335cb352029d0d"
server_key_version: "encryptionkey@0"
I20260621 07:02:59.973438 23858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:02:59.977181 23873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:02:59.978670 23858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:02:59.978874 23858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "c73db7149435481b803fdc4ea6d47e78"
format_stamp: "Formatted at 2026-06-21 07:02:59 on dist-test-slave-9zdv"
server_key: "c7322cb35a90bf095e2491d6abe2addd"
server_key_iv: "c30c01a757ad4e7c92335cb352029d0d"
server_key_version: "encryptionkey@0"
I20260621 07:02:59.979142 23858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:02:59.989859 23858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:02:59.994395 23858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:02:59.994725 23858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:00.012944 23858 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42917
I20260621 07:03:00.012966 23924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42917 every 8 connection(s)
I20260621 07:03:00.014464 23858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:00.015529 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23858
I20260621 07:03:00.015723 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:00.016212 3031 external_mini_cluster.cc:1428] Setting key ed18069970ba9523740ebbfc81c887f7
I20260621 07:03:00.018843 23925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:00.030006 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:00.018277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60127 (local address 127.0.0.1:42917)
0621 07:03:00.019139 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:00.019154 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:00.019200 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:03:00.020807 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:00.020841 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:00.021249 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:00.021590 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:00.022219 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:00.023657 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:00.025533 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:00.026210 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:00.026360 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:00.027703 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:00.027710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:00.027726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:00.028268 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:00.028341 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:00.028375 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:00.028461 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:00.028741 (+ 280us) server_negotiation.cc:299] Negotiation successful
0621 07:03:00.029033 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":387,"threads_started":1}
I20260621 07:03:00.031266 23925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78: Bootstrap starting.
I20260621 07:03:00.034092 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:00.035022 23925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:00.036217 23925 log.cc:824] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:00.039577 23925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78: No bootstrap required, opened a new log
I20260621 07:03:00.045851 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:00.039217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40485 (local address 127.0.0.1:42917)
0621 07:03:00.039402 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:00.039408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:00.039889 (+ 481us) server_negotiation.cc:407] Connection header received
0621 07:03:00.040037 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:00.040043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:00.040131 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:00.040247 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:00.040665 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:00.041780 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:00.043208 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:00.043911 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:00.044049 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:00.045075 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:00.045086 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:00.045091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:00.045110 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:00.045153 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:00.045177 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:00.045281 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:00.045410 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 07:03:00.045500 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 07:03:00.049134 23925 raft_consensus.cc:348] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } }
I20260621 07:03:00.049487 23925 raft_consensus.cc:374] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:00.049561 23925 raft_consensus.cc:729] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73db7149435481b803fdc4ea6d47e78, State: Initialized, Role: FOLLOWER
I20260621 07:03:00.050055 23925 consensus_queue.cc:260] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } }
I20260621 07:03:00.050256 23925 raft_consensus.cc:388] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:00.050355 23925 raft_consensus.cc:482] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:00.050475 23925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:00.052925 23925 raft_consensus.cc:504] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } }
I20260621 07:03:00.053432 23925 leader_election.cc:302] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c73db7149435481b803fdc4ea6d47e78; no voters:
I20260621 07:03:00.053984 23925 leader_election.cc:288] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:00.054147 23935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:00.054842 23935 raft_consensus.cc:686] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [term 1 LEADER]: Becoming Leader. State: Replica: c73db7149435481b803fdc4ea6d47e78, State: Running, Role: LEADER
I20260621 07:03:00.055446 23935 consensus_queue.cc:237] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } }
I20260621 07:03:00.056183 23925 sys_catalog.cc:564] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:00.058373 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c73db7149435481b803fdc4ea6d47e78. Latest consensus state: current_term: 1 leader_uuid: "c73db7149435481b803fdc4ea6d47e78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } } }
I20260621 07:03:00.058796 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:00.059110 23936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c73db7149435481b803fdc4ea6d47e78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73db7149435481b803fdc4ea6d47e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42917 } } }
I20260621 07:03:00.059453 23936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:00.064803 23941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:00.070235 23941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:00.078775 23941 catalog_manager.cc:1347] Generated new cluster ID: de385e3b26f94a4ebb9cd8e316426323
I20260621 07:03:00.078892 23941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:00.096220 23941 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:00.097501 23941 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:00.106843 23941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c73db7149435481b803fdc4ea6d47e78: Generated new TSK 0
I20260621 07:03:00.107702 23941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:00.121393 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23858
W20260621 07:03:00.122469 23920 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60127
W20260621 07:03:00.123198 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23858: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:00.123355 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23858
2026-06-21T07:03:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:00 dist-test-slave-9zdv krb5kdc[23956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:00 dist-test-slave-9zdv krb5kdc[23956](info): set up 2 sockets
Jun 21 07:03:00 dist-test-slave-9zdv krb5kdc[23956](info): commencing operation
krb5kdc: starting...
W20260621 07:03:01.243669 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:01 dist-test-slave-9zdv krb5kdc[23956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:01.508277 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:01.846871 23969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:01.847299 23969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:01.847473 23969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:01.855062 23969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:01.855201 23969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:01.855290 23969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:01.855396 23969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:01.855479 23969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:01.867383 23969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:01.868705 23969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:01.870030 23969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:01.882570 23974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:01.882625 23977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:01.882598 23975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:01.883131 23969 server_base.cc:1034] running on GCE node
I20260621 07:03:01.884138 23969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:01.885637 23969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:01.886879 23969 hybrid_clock.cc:648] HybridClock initialized: now 1782025381886836 us; error 37 us; skew 500 ppm
I20260621 07:03:01.887257 23969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:01 dist-test-slave-9zdv krb5kdc[23956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:01.895073 23969 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:01.896756 23969 webserver.cc:460] Webserver started at http://127.0.0.1:44051/ using document root <none> and password file <none>
I20260621 07:03:01.897511 23969 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:01.897667 23969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:01.897989 23969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:01.901113 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "c2c695c6159049f2a4d78190bf5676e5"
format_stamp: "Formatted at 2026-06-21 07:03:01 on dist-test-slave-9zdv"
server_key: "00f8a1c68a7fcfdd04ccf715034208a0"
server_key_iv: "19a8c0cff6045776dcf14d58fed50287"
server_key_version: "encryptionkey@0"
I20260621 07:03:01.901934 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "c2c695c6159049f2a4d78190bf5676e5"
format_stamp: "Formatted at 2026-06-21 07:03:01 on dist-test-slave-9zdv"
server_key: "00f8a1c68a7fcfdd04ccf715034208a0"
server_key_iv: "19a8c0cff6045776dcf14d58fed50287"
server_key_version: "encryptionkey@0"
I20260621 07:03:01.907442 23969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260621 07:03:01.911187 23984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:01.912580 23969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:03:01.912773 23969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "c2c695c6159049f2a4d78190bf5676e5"
format_stamp: "Formatted at 2026-06-21 07:03:01 on dist-test-slave-9zdv"
server_key: "00f8a1c68a7fcfdd04ccf715034208a0"
server_key_iv: "19a8c0cff6045776dcf14d58fed50287"
server_key_version: "encryptionkey@0"
I20260621 07:03:01.912984 23969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:01.919029 23969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:01.923348 23969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:01.923681 23969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:01.942556 23969 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39071
I20260621 07:03:01.942590 24035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39071 every 8 connection(s)
I20260621 07:03:01.944031 23969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:01.944384 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 23969
I20260621 07:03:01.944563 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:01.945037 3031 external_mini_cluster.cc:1428] Setting key 2ad28beca055e5f72ee6dd3f2968228a
I20260621 07:03:01.948959 24036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:01.953896 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:01.946979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57467 (local address 127.0.0.1:39071)
0621 07:03:01.947779 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:01.947801 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:01.947855 (+ 54us) server_negotiation.cc:407] Connection header received
0621 07:03:01.950119 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:01.950146 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:01.950572 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:01.951005 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:01.951966 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:01.951976 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:01.951998 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:01.952014 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:01.952104 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:01.952135 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:01.952357 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:01.952486 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 07:03:01.952688 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":312,"threads_started":1}
I20260621 07:03:01.957818 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:01.961934 24036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5: Bootstrap starting.
I20260621 07:03:01.965031 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:01.962215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38911 (local address 127.0.0.1:39071)
0621 07:03:01.962453 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:01.962462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:01.962789 (+ 327us) server_negotiation.cc:407] Connection header received
0621 07:03:01.963018 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:01.963024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:01.963083 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:01.963206 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:01.964337 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:01.964343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:01.964348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:01.964369 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:01.964415 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:01.964431 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:01.964565 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:01.964689 (+ 124us) server_negotiation.cc:299] Negotiation successful
0621 07:03:01.964748 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:03:01.965926 24036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:01.967258 24036 log.cc:824] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:01.969930 24036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5: No bootstrap required, opened a new log
I20260621 07:03:01.977890 24036 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260621 07:03:01.978247 24036 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:01.978312 24036 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c695c6159049f2a4d78190bf5676e5, State: Initialized, Role: FOLLOWER
I20260621 07:03:01.978904 24036 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260621 07:03:01.979166 24036 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:01.979274 24036 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:01.979400 24036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:01.981602 24036 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260621 07:03:01.982237 24036 leader_election.cc:302] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2c695c6159049f2a4d78190bf5676e5; no voters:
I20260621 07:03:01.982887 24036 leader_election.cc:288] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:01.983067 24046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:01.983789 24046 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [term 1 LEADER]: Becoming Leader. State: Replica: c2c695c6159049f2a4d78190bf5676e5, State: Running, Role: LEADER
I20260621 07:03:01.984371 24046 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260621 07:03:01.985076 24036 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:01.987313 24047 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2c695c6159049f2a4d78190bf5676e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260621 07:03:01.988041 24047 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:01.987330 24048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2c695c6159049f2a4d78190bf5676e5. Latest consensus state: current_term: 1 leader_uuid: "c2c695c6159049f2a4d78190bf5676e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2c695c6159049f2a4d78190bf5676e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260621 07:03:01.988624 24048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:01.989753 24053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:01.995672 24053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:02.005472 24053 catalog_manager.cc:1347] Generated new cluster ID: 9e9880707b14451f9f8c5a45a9d1d04a
I20260621 07:03:02.005599 24053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:02.019476 24053 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:02.019979 24053 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:02.031823 24053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2c695c6159049f2a4d78190bf5676e5: Generated new TSK 0
I20260621 07:03:02.032759 24053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:02.049409 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23969
W20260621 07:03:02.050525 24031 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57467
W20260621 07:03:02.051409 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 23969: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:02.051566 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 23969
2026-06-21T07:03:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:02 dist-test-slave-9zdv krb5kdc[24067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:02 dist-test-slave-9zdv krb5kdc[24067](info): set up 2 sockets
Jun 21 07:03:02 dist-test-slave-9zdv krb5kdc[24067](info): commencing operation
krb5kdc: starting...
W20260621 07:03:03.171725 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:03 dist-test-slave-9zdv krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:03.434790 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:03.772845 24080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:03.773129 24080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:03.773222 24080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:03.780520 24080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:03.780639 24080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:03.780689 24080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:03.780747 24080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:03.780791 24080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:03.790786 24080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:03.792165 24080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:03.793471 24080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:03.805827 24086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:03.805888 24085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:03.805961 24088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:03.806411 24080 server_base.cc:1034] running on GCE node
I20260621 07:03:03.807382 24080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:03.808837 24080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:03.810096 24080 hybrid_clock.cc:648] HybridClock initialized: now 1782025383810061 us; error 51 us; skew 500 ppm
I20260621 07:03:03.810497 24080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:03 dist-test-slave-9zdv krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:03.813868 24080 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:03.815454 24080 webserver.cc:460] Webserver started at http://127.0.0.1:35473/ using document root <none> and password file <none>
I20260621 07:03:03.816183 24080 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:03.816339 24080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:03.816650 24080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:03.819841 24080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "684267f4b96c44a9af7cdd137546c30f"
format_stamp: "Formatted at 2026-06-21 07:03:03 on dist-test-slave-9zdv"
server_key: "39e99e7429ae21b0f8fff073e09da87b"
server_key_iv: "e539f98588153c7bcf7619c3ca591833"
server_key_version: "encryptionkey@0"
I20260621 07:03:03.820604 24080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "684267f4b96c44a9af7cdd137546c30f"
format_stamp: "Formatted at 2026-06-21 07:03:03 on dist-test-slave-9zdv"
server_key: "39e99e7429ae21b0f8fff073e09da87b"
server_key_iv: "e539f98588153c7bcf7619c3ca591833"
server_key_version: "encryptionkey@0"
I20260621 07:03:03.826323 24080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 07:03:03.829874 24095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:03.831259 24080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:03:03.831449 24080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "684267f4b96c44a9af7cdd137546c30f"
format_stamp: "Formatted at 2026-06-21 07:03:03 on dist-test-slave-9zdv"
server_key: "39e99e7429ae21b0f8fff073e09da87b"
server_key_iv: "e539f98588153c7bcf7619c3ca591833"
server_key_version: "encryptionkey@0"
I20260621 07:03:03.831663 24080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:03.848618 24080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:03.853235 24080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:03.853547 24080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:03.873493 24080 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45509
I20260621 07:03:03.873551 24146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45509 every 8 connection(s)
I20260621 07:03:03.875011 24080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:03.878747 24147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:03.881493 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24080
I20260621 07:03:03.881677 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:03.882336 3031 external_mini_cluster.cc:1428] Setting key 13c3b45e03840b9ad2d5da59cab78251
I20260621 07:03:03.891618 24147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f: Bootstrap starting.
I20260621 07:03:03.895550 24147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:03.896869 24147 log.cc:824] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f: Log is configured to *not* fsync() on all Append() calls
Jun 21 07:03:03 dist-test-slave-9zdv krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 07:03:03.899850 24147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f: No bootstrap required, opened a new log
I20260621 07:03:03.907158 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:03.884223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:37211 (local address 127.0.0.1:45509)
0621 07:03:03.885076 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:03.885087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:03.885128 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:03:03.886756 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:03.886779 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:03.887344 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:03.887678 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:03.889445 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:03.890711 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:03.892582 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:03.893256 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:03.893394 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:03.898316 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:03.898324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:03.898338 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:03.898376 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:03.902163 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:03.903099 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:03.903108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:03.903117 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:03.903361 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:03.903817 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:03.903821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:03.903825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:03.904542 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:03.904864 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:03.905282 (+ 418us) server_negotiation.cc:299] Negotiation successful
0621 07:03:03.905649 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":389,"threads_started":1}
I20260621 07:03:03.910024 24147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } }
I20260621 07:03:03.910447 24147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:03.910624 24147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 684267f4b96c44a9af7cdd137546c30f, State: Initialized, Role: FOLLOWER
I20260621 07:03:03.911316 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:03.911324 24147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } }
I20260621 07:03:03.911586 24147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:03.911715 24147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:03.911882 24147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:03.915202 24147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } }
I20260621 07:03:03.915861 24147 leader_election.cc:302] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 684267f4b96c44a9af7cdd137546c30f; no voters:
I20260621 07:03:03.916544 24147 leader_election.cc:288] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:03.916716 24156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:03.920840 24156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [term 1 LEADER]: Becoming Leader. State: Replica: 684267f4b96c44a9af7cdd137546c30f, State: Running, Role: LEADER
I20260621 07:03:03.921404 24156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } }
I20260621 07:03:03.923388 24156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 684267f4b96c44a9af7cdd137546c30f. Latest consensus state: current_term: 1 leader_uuid: "684267f4b96c44a9af7cdd137546c30f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } } }
I20260621 07:03:03.923707 24156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:03.925771 24147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:03.927549 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "684267f4b96c44a9af7cdd137546c30f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684267f4b96c44a9af7cdd137546c30f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45509 } } }
I20260621 07:03:03.927969 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:03.929643 24161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:03.935849 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:03.916822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:49697 (local address 127.0.0.1:45509)
0621 07:03:03.917023 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:03.917035 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:03.917723 (+ 688us) server_negotiation.cc:407] Connection header received
0621 07:03:03.917933 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:03.917941 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:03.918050 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:03.918184 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:03.920556 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:03.922157 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:03.924102 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:03.925035 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:03.925173 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:03.928664 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:03.928670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:03.928673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:03.928704 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:03.931863 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:03.932903 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:03.932908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:03.932911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:03.933066 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:03.933709 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:03.933715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:03.933719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:03.934195 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:03.934498 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:03.935301 (+ 803us) server_negotiation.cc:299] Negotiation successful
0621 07:03:03.935486 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:03:03.936579 24161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:03.948712 24161 catalog_manager.cc:1347] Generated new cluster ID: 6d7f491e4910433b8217d1b2b4ef09e0
I20260621 07:03:03.948829 24161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:03.974462 24161 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:03.975812 24161 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:03.991700 24161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 684267f4b96c44a9af7cdd137546c30f: Generated new TSK 0
I20260621 07:03:03.992811 24161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:04.007351 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24080
I20260621 07:03:04.008553 24142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:04.102146 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24080
W20260621 07:03:04.119385 24065 connection.cc:535] client connection to 127.0.0.1:45509 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45509): Connection reset by peer (error 104)
2026-06-21T07:03:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:04 dist-test-slave-9zdv krb5kdc[24179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:04 dist-test-slave-9zdv krb5kdc[24179](info): set up 2 sockets
Jun 21 07:03:04 dist-test-slave-9zdv krb5kdc[24179](info): commencing operation
krb5kdc: starting...
W20260621 07:03:05.219702 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:05 dist-test-slave-9zdv krb5kdc[24179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:05.486513 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:05.826357 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:05.826725 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:05.826913 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:05.834445 24192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:05.834610 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:05.834717 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:05.834830 24192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:05.834918 24192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:05.844995 24192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:05.846356 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:05.847808 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:05.859578 24197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:05.859549 24198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:05.859637 24200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:05.860836 24192 server_base.cc:1034] running on GCE node
I20260621 07:03:05.861547 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:05.862905 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:05.864181 24192 hybrid_clock.cc:648] HybridClock initialized: now 1782025385864143 us; error 34 us; skew 500 ppm
I20260621 07:03:05.864575 24192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:05 dist-test-slave-9zdv krb5kdc[24179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:05.867756 24192 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:05.869287 24192 webserver.cc:460] Webserver started at http://127.0.0.1:38813/ using document root <none> and password file <none>
I20260621 07:03:05.869946 24192 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:05.870044 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:05.870298 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:05.873318 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "99ef10fd87ab4427963aee183c73233f"
format_stamp: "Formatted at 2026-06-21 07:03:05 on dist-test-slave-9zdv"
server_key: "9bb246dbad7be1768c91e31f99857ee4"
server_key_iv: "d3767a0b72539d35dcb5391b5e7cf6c9"
server_key_version: "encryptionkey@0"
I20260621 07:03:05.874028 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "99ef10fd87ab4427963aee183c73233f"
format_stamp: "Formatted at 2026-06-21 07:03:05 on dist-test-slave-9zdv"
server_key: "9bb246dbad7be1768c91e31f99857ee4"
server_key_iv: "d3767a0b72539d35dcb5391b5e7cf6c9"
server_key_version: "encryptionkey@0"
I20260621 07:03:05.880071 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260621 07:03:05.884133 24207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:05.885524 24192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260621 07:03:05.885717 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "99ef10fd87ab4427963aee183c73233f"
format_stamp: "Formatted at 2026-06-21 07:03:05 on dist-test-slave-9zdv"
server_key: "9bb246dbad7be1768c91e31f99857ee4"
server_key_iv: "d3767a0b72539d35dcb5391b5e7cf6c9"
server_key_version: "encryptionkey@0"
I20260621 07:03:05.885921 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:05.897351 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:05.901324 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:05.901643 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:05.920713 24192 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38889
I20260621 07:03:05.920734 24258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38889 every 8 connection(s)
I20260621 07:03:05.922155 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:05.922632 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24192
I20260621 07:03:05.922820 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:05.923475 3031 external_mini_cluster.cc:1428] Setting key b1986cf18751cb5ca6bbc935b3af54ce
I20260621 07:03:05.926733 24259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:05.939952 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:05.925391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:34055 (local address 127.0.0.1:38889)
0621 07:03:05.926413 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:05.926434 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:05.927058 (+ 624us) server_negotiation.cc:407] Connection header received
0621 07:03:05.929512 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:05.929539 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:05.930064 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:05.930517 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:05.931050 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:05.933101 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:05.934437 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:05.935548 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:05.935755 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:05.936987 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:05.936997 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:05.937016 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:05.937814 (+ 798us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:05.937911 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:05.937943 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:05.938116 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:05.938396 (+ 280us) server_negotiation.cc:299] Negotiation successful
0621 07:03:05.938588 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":390,"threads_started":1}
I20260621 07:03:05.941536 24259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f: Bootstrap starting.
I20260621 07:03:05.944180 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:05.945199 24259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:05.946152 24259 log.cc:824] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:05.949383 24259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f: No bootstrap required, opened a new log
I20260621 07:03:05.956302 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:05.948813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:40843 (local address 127.0.0.1:38889)
0621 07:03:05.949052 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:05.949061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:05.949501 (+ 440us) server_negotiation.cc:407] Connection header received
0621 07:03:05.949680 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:05.949684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:05.949747 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:05.949843 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:05.950489 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:05.951355 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:05.953356 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:05.953880 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:05.953986 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:05.955297 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:05.955302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:05.955305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:05.955328 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:05.955408 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:05.955427 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:05.955524 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:05.955840 (+ 316us) server_negotiation.cc:299] Negotiation successful
0621 07:03:05.955912 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260621 07:03:05.959462 24259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260621 07:03:05.959959 24259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:05.960079 24259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ef10fd87ab4427963aee183c73233f, State: Initialized, Role: FOLLOWER
I20260621 07:03:05.960758 24259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260621 07:03:05.960994 24259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:05.961169 24259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:05.961347 24259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:05.963686 24259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260621 07:03:05.964203 24259 leader_election.cc:302] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99ef10fd87ab4427963aee183c73233f; no voters:
I20260621 07:03:05.964841 24259 leader_election.cc:288] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:05.964995 24269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:05.966147 24269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [term 1 LEADER]: Becoming Leader. State: Replica: 99ef10fd87ab4427963aee183c73233f, State: Running, Role: LEADER
I20260621 07:03:05.966652 24269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260621 07:03:05.967135 24259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:05.969919 24271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ef10fd87ab4427963aee183c73233f. Latest consensus state: current_term: 1 leader_uuid: "99ef10fd87ab4427963aee183c73233f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260621 07:03:05.970350 24271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:05.970039 24270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99ef10fd87ab4427963aee183c73233f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ef10fd87ab4427963aee183c73233f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260621 07:03:05.971311 24270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:05.973623 24278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:05.978540 24278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:05.989076 24278 catalog_manager.cc:1347] Generated new cluster ID: 8404e3bd8166446cbc333432244976ee
I20260621 07:03:05.989257 24278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:06.003509 24278 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:06.005439 24278 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:06.015856 24278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ef10fd87ab4427963aee183c73233f: Generated new TSK 0
I20260621 07:03:06.016980 24278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:06.024971 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24192
W20260621 07:03:06.026089 24254 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.92:34055
W20260621 07:03:06.026888 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24192: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:06.027087 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24192
W20260621 07:03:06.044476 24177 connection.cc:535] client connection to 127.0.0.1:38889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38889): Connection reset by peer (error 104)
2026-06-21T07:03:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1925 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:06 dist-test-slave-9zdv krb5kdc[24290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:06 dist-test-slave-9zdv krb5kdc[24290](info): set up 2 sockets
Jun 21 07:03:06 dist-test-slave-9zdv krb5kdc[24290](info): commencing operation
krb5kdc: starting...
W20260621 07:03:07.131577 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:07 dist-test-slave-9zdv krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:07.393795 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:07.736740 24303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:07.737104 24303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:07.737274 24303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:07.744671 24303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:07.744853 24303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:07.745118 24303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:07.745266 24303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:07.745398 24303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:07.755841 24303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:07.757208 24303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:07.758572 24303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:07.770778 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:07.770870 24308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:07.770931 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:07.771920 24303 server_base.cc:1034] running on GCE node
I20260621 07:03:07.772809 24303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:07.774250 24303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:07.775467 24303 hybrid_clock.cc:648] HybridClock initialized: now 1782025387775427 us; error 31 us; skew 500 ppm
I20260621 07:03:07.775846 24303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:07 dist-test-slave-9zdv krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:07.782416 24303 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:07.783985 24303 webserver.cc:460] Webserver started at http://127.0.0.1:42609/ using document root <none> and password file <none>
I20260621 07:03:07.784708 24303 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:07.784844 24303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:07.785152 24303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:07.788767 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "b4736749d45e476485c3bc0ff26e1625"
format_stamp: "Formatted at 2026-06-21 07:03:07 on dist-test-slave-9zdv"
server_key: "1f46b4a3081a5edf2a27f713aeeb3815"
server_key_iv: "6337e0c55633c28c49ecbb39665c1f97"
server_key_version: "encryptionkey@0"
I20260621 07:03:07.789559 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "b4736749d45e476485c3bc0ff26e1625"
format_stamp: "Formatted at 2026-06-21 07:03:07 on dist-test-slave-9zdv"
server_key: "1f46b4a3081a5edf2a27f713aeeb3815"
server_key_iv: "6337e0c55633c28c49ecbb39665c1f97"
server_key_version: "encryptionkey@0"
I20260621 07:03:07.794799 24303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260621 07:03:07.798457 24318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:07.799957 24303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:03:07.800160 24303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "b4736749d45e476485c3bc0ff26e1625"
format_stamp: "Formatted at 2026-06-21 07:03:07 on dist-test-slave-9zdv"
server_key: "1f46b4a3081a5edf2a27f713aeeb3815"
server_key_iv: "6337e0c55633c28c49ecbb39665c1f97"
server_key_version: "encryptionkey@0"
I20260621 07:03:07.800381 24303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:07.806308 24303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:07.810501 24303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:07.810817 24303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:07.829680 24303 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42037
I20260621 07:03:07.829727 24369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42037 every 8 connection(s)
I20260621 07:03:07.831195 24303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:07.835042 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:07.839740 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24303
I20260621 07:03:07.839918 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:07.840487 3031 external_mini_cluster.cc:1428] Setting key 356c9e89223074f5000ddd3984c1123f
I20260621 07:03:07.847002 24370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625: Bootstrap starting.
I20260621 07:03:07.848933 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:07.842441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:41401 (local address 127.0.0.1:42037)
0621 07:03:07.843319 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:07.843331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:07.843371 (+ 40us) server_negotiation.cc:407] Connection header received
0621 07:03:07.845420 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:07.845444 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:07.845793 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:07.846128 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:07.847302 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:07.847310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:07.847327 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:07.847343 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:07.847407 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:07.847426 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:07.847596 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:07.847768 (+ 172us) server_negotiation.cc:299] Negotiation successful
0621 07:03:07.847985 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":405,"threads_started":1}
I20260621 07:03:07.850688 24370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:07.851986 24370 log.cc:824] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:07.852861 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:07.855298 24370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625: No bootstrap required, opened a new log
I20260621 07:03:07.860708 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:07.858079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:50741 (local address 127.0.0.1:42037)
0621 07:03:07.858301 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:07.858307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:07.858697 (+ 390us) server_negotiation.cc:407] Connection header received
0621 07:03:07.858855 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:07.858859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:07.858905 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:07.859061 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:07.860035 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:07.860038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:07.860042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:07.860060 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:07.860097 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:07.860111 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:07.860201 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:07.860337 (+ 136us) server_negotiation.cc:299] Negotiation successful
0621 07:03:07.860390 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260621 07:03:07.864629 24370 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260621 07:03:07.865015 24370 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:07.865176 24370 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4736749d45e476485c3bc0ff26e1625, State: Initialized, Role: FOLLOWER
I20260621 07:03:07.865777 24370 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260621 07:03:07.866011 24370 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:07.866166 24370 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:07.866340 24370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:07.868566 24370 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260621 07:03:07.869156 24370 leader_election.cc:302] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4736749d45e476485c3bc0ff26e1625; no voters:
I20260621 07:03:07.869783 24370 leader_election.cc:288] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:07.869967 24380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:07.871191 24380 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [term 1 LEADER]: Becoming Leader. State: Replica: b4736749d45e476485c3bc0ff26e1625, State: Running, Role: LEADER
I20260621 07:03:07.871791 24380 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260621 07:03:07.872092 24370 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:07.874516 24382 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4736749d45e476485c3bc0ff26e1625. Latest consensus state: current_term: 1 leader_uuid: "b4736749d45e476485c3bc0ff26e1625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } } }
I20260621 07:03:07.874917 24382 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:07.875013 24381 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4736749d45e476485c3bc0ff26e1625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4736749d45e476485c3bc0ff26e1625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } } }
I20260621 07:03:07.875385 24381 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:07.879966 24387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:07.885361 24387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:07.903247 24387 catalog_manager.cc:1347] Generated new cluster ID: 5a4767c57fcb4303a329807114ed3754
I20260621 07:03:07.903429 24387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:07.921936 24387 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:07.922425 24387 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:07.932276 24387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4736749d45e476485c3bc0ff26e1625: Generated new TSK 0
I20260621 07:03:07.933140 24387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:07.939877 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24303
W20260621 07:03:07.941002 24365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.92:41401
W20260621 07:03:07.941776 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24303: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:07.941926 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24303
2026-06-21T07:03:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:08 dist-test-slave-9zdv krb5kdc[24401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:08 dist-test-slave-9zdv krb5kdc[24401](info): set up 2 sockets
Jun 21 07:03:08 dist-test-slave-9zdv krb5kdc[24401](info): commencing operation
krb5kdc: starting...
W20260621 07:03:09.039783 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:09 dist-test-slave-9zdv krb5kdc[24401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:09.299983 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:09.636999 24414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:09.637310 24414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:09.637503 24414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:09.645052 24414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:09.645176 24414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:09.645284 24414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:09.645412 24414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:09.645510 24414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:09.655539 24414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:09.656936 24414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:09.658277 24414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:09.670588 24419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:09.670570 24422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:09.670559 24420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:09.671579 24414 server_base.cc:1034] running on GCE node
I20260621 07:03:09.672400 24414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:09.673844 24414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:09.675050 24414 hybrid_clock.cc:648] HybridClock initialized: now 1782025389675006 us; error 48 us; skew 500 ppm
I20260621 07:03:09.675365 24414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:09 dist-test-slave-9zdv krb5kdc[24401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:09.678548 24414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:09.680191 24414 webserver.cc:460] Webserver started at http://127.0.0.1:45077/ using document root <none> and password file <none>
I20260621 07:03:09.680923 24414 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:09.681066 24414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:09.681401 24414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:09.684579 24414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "dccf937e647241e2a3aa71ce56126a7e"
format_stamp: "Formatted at 2026-06-21 07:03:09 on dist-test-slave-9zdv"
server_key: "0b398322569f9b6d3eb3fc6a21d46755"
server_key_iv: "4b488da066631a81ce9ed0b9aca096c5"
server_key_version: "encryptionkey@0"
I20260621 07:03:09.685364 24414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "dccf937e647241e2a3aa71ce56126a7e"
format_stamp: "Formatted at 2026-06-21 07:03:09 on dist-test-slave-9zdv"
server_key: "0b398322569f9b6d3eb3fc6a21d46755"
server_key_iv: "4b488da066631a81ce9ed0b9aca096c5"
server_key_version: "encryptionkey@0"
I20260621 07:03:09.695255 24414 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.007s
I20260621 07:03:09.700202 24429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:09.701612 24414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260621 07:03:09.701830 24414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "dccf937e647241e2a3aa71ce56126a7e"
format_stamp: "Formatted at 2026-06-21 07:03:09 on dist-test-slave-9zdv"
server_key: "0b398322569f9b6d3eb3fc6a21d46755"
server_key_iv: "4b488da066631a81ce9ed0b9aca096c5"
server_key_version: "encryptionkey@0"
I20260621 07:03:09.702061 24414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:09.742080 24414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:09.746428 24414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:09.746745 24414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:09.765933 24414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41719
I20260621 07:03:09.766047 24480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41719 every 8 connection(s)
I20260621 07:03:09.767473 24414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:09.771224 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:09.776506 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24414
I20260621 07:03:09.776684 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:09.777245 3031 external_mini_cluster.cc:1428] Setting key 2113a9087cb5b1471499d6400bfe4d7f
I20260621 07:03:09.782778 24481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e: Bootstrap starting.
I20260621 07:03:09.786685 24481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:09.787977 24481 log.cc:824] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:09.790920 24481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e: No bootstrap required, opened a new log
Jun 21 07:03:09 dist-test-slave-9zdv krb5kdc[24401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 07:03:09.801836 24481 raft_consensus.cc:348] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } }
I20260621 07:03:09.802320 24481 raft_consensus.cc:374] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:09.802377 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:09.779262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:42803 (local address 127.0.0.1:41719)
0621 07:03:09.780109 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:09.780121 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:09.780166 (+ 45us) server_negotiation.cc:407] Connection header received
0621 07:03:09.781868 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:09.781893 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:09.782300 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:09.782687 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:09.784494 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:09.785839 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:09.787672 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:09.788427 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:09.788671 (+ 244us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:09.792981 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:09.792992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:09.793013 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:09.793068 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:09.797322 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:09.798208 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:09.798217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:09.798226 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:09.798456 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:09.798925 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:09.798959 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:09.798965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:09.799803 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:09.800185 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:09.800528 (+ 343us) server_negotiation.cc:299] Negotiation successful
0621 07:03:09.800915 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":370,"threads_started":1}
I20260621 07:03:09.802510 24481 raft_consensus.cc:729] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dccf937e647241e2a3aa71ce56126a7e, State: Initialized, Role: FOLLOWER
I20260621 07:03:09.803220 24481 consensus_queue.cc:260] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } }
I20260621 07:03:09.803524 24481 raft_consensus.cc:388] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:09.803669 24481 raft_consensus.cc:482] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:09.803855 24481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:09.806648 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:09.806744 24481 raft_consensus.cc:504] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } }
I20260621 07:03:09.807338 24481 leader_election.cc:302] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dccf937e647241e2a3aa71ce56126a7e; no voters:
I20260621 07:03:09.807940 24481 leader_election.cc:288] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:09.808095 24486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:09.809924 24481 sys_catalog.cc:564] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:09.811218 24486 raft_consensus.cc:686] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [term 1 LEADER]: Becoming Leader. State: Replica: dccf937e647241e2a3aa71ce56126a7e, State: Running, Role: LEADER
I20260621 07:03:09.812053 24486 consensus_queue.cc:237] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } }
I20260621 07:03:09.815675 24487 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dccf937e647241e2a3aa71ce56126a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } } }
I20260621 07:03:09.816100 24487 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:09.815347 24488 sys_catalog.cc:455] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dccf937e647241e2a3aa71ce56126a7e. Latest consensus state: current_term: 1 leader_uuid: "dccf937e647241e2a3aa71ce56126a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dccf937e647241e2a3aa71ce56126a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41719 } } }
I20260621 07:03:09.816869 24488 sys_catalog.cc:458] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:09.818990 24495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:09.828428 24495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:09.847406 24495 catalog_manager.cc:1347] Generated new cluster ID: fbf0db72f168441d9aef1ac186ab8927
I20260621 07:03:09.847525 24495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:09.855927 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:09.824695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:58373 (local address 127.0.0.1:41719)
0621 07:03:09.824892 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:09.824902 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:09.835697 (+ 10795us) server_negotiation.cc:407] Connection header received
0621 07:03:09.836088 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:09.836101 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:09.836209 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:09.836299 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:09.839041 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:09.840616 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:09.843060 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:09.844010 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:09.844141 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:09.847856 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:09.847862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:09.847867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:09.847915 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:09.851623 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:09.852664 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:09.852669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:09.852673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:09.852809 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:09.853368 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:09.853373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:09.853377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:09.853819 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:09.854094 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:09.855352 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0621 07:03:09.855535 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:03:09.856600 24495 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:09.858425 24495 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:09.869542 24495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dccf937e647241e2a3aa71ce56126a7e: Generated new TSK 0
I20260621 07:03:09.870415 24495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:09.878818 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24414
I20260621 07:03:09.880537 24476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:09.978168 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24414
W20260621 07:03:09.992451 24399 connection.cc:535] client connection to 127.0.0.1:41719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41719): Connection reset by peer (error 104)
2026-06-21T07:03:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:10 dist-test-slave-9zdv krb5kdc[24514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:10 dist-test-slave-9zdv krb5kdc[24514](info): set up 2 sockets
Jun 21 07:03:10 dist-test-slave-9zdv krb5kdc[24514](info): commencing operation
krb5kdc: starting...
W20260621 07:03:11.099716 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:11 dist-test-slave-9zdv krb5kdc[24514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:11.363509 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:11.709257 24527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:11.709589 24527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:11.709753 24527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:11.717234 24527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:11.717366 24527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:11.717499 24527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:11.717607 24527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:11.717679 24527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:11.727872 24527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:11.729209 24527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:11.730551 24527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:11.742862 24533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:11.742990 24535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:11.742972 24532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:11.743964 24527 server_base.cc:1034] running on GCE node
I20260621 07:03:11.744771 24527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:11.746390 24527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:11.747632 24527 hybrid_clock.cc:648] HybridClock initialized: now 1782025391747589 us; error 39 us; skew 500 ppm
I20260621 07:03:11.748000 24527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:11 dist-test-slave-9zdv krb5kdc[24514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:11.751286 24527 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:11.752815 24527 webserver.cc:460] Webserver started at http://127.0.0.1:39073/ using document root <none> and password file <none>
I20260621 07:03:11.753480 24527 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:11.753597 24527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:11.753870 24527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:11.757097 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "c083f66936d14d78a9dee19cd2d2b40a"
format_stamp: "Formatted at 2026-06-21 07:03:11 on dist-test-slave-9zdv"
server_key: "72f98ac33e7857b4fb6bffa523df0140"
server_key_iv: "08795bb515eb41e15ef3362344df9a6c"
server_key_version: "encryptionkey@0"
I20260621 07:03:11.757891 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "c083f66936d14d78a9dee19cd2d2b40a"
format_stamp: "Formatted at 2026-06-21 07:03:11 on dist-test-slave-9zdv"
server_key: "72f98ac33e7857b4fb6bffa523df0140"
server_key_iv: "08795bb515eb41e15ef3362344df9a6c"
server_key_version: "encryptionkey@0"
I20260621 07:03:11.763459 24527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260621 07:03:11.767117 24542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:11.768736 24527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:03:11.768961 24527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "c083f66936d14d78a9dee19cd2d2b40a"
format_stamp: "Formatted at 2026-06-21 07:03:11 on dist-test-slave-9zdv"
server_key: "72f98ac33e7857b4fb6bffa523df0140"
server_key_iv: "08795bb515eb41e15ef3362344df9a6c"
server_key_version: "encryptionkey@0"
I20260621 07:03:11.769222 24527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:11.803002 24527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:11.807178 24527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:11.807466 24527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:11.826159 24527 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40999
I20260621 07:03:11.826205 24593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40999 every 8 connection(s)
I20260621 07:03:11.827605 24527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:11.830905 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24527
I20260621 07:03:11.831142 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:11.831523 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:11.831589 3031 external_mini_cluster.cc:1428] Setting key 58d3a0e914527d9ed141d58f09f52b6a
I20260621 07:03:11.845737 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:11.833420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:50651 (local address 127.0.0.1:40999)
0621 07:03:11.834228 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:11.834238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:11.834300 (+ 62us) server_negotiation.cc:407] Connection header received
0621 07:03:11.836278 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:11.836301 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:11.836748 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:11.837105 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:11.837752 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:11.839158 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:11.841048 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:11.841720 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:11.841871 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:11.843339 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:11.843347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:11.843362 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:11.843944 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:11.844035 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:11.844061 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:11.844176 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:11.844469 (+ 293us) server_negotiation.cc:299] Negotiation successful
0621 07:03:11.844660 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":350,"threads_started":1}
I20260621 07:03:11.846072 24594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a: Bootstrap starting.
I20260621 07:03:11.849843 24594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:11.849869 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:11.850872 24594 log.cc:824] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:11.854054 24594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a: No bootstrap required, opened a new log
I20260621 07:03:11.861001 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:11.854676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:50427 (local address 127.0.0.1:40999)
0621 07:03:11.854825 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:11.854829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:11.855343 (+ 514us) server_negotiation.cc:407] Connection header received
0621 07:03:11.855574 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:11.855579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:11.855638 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:11.855740 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:11.856227 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:11.857040 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:11.858351 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:11.858848 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:11.859076 (+ 228us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:11.860243 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:11.860249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:11.860252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:11.860269 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:11.860313 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:11.860331 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:11.860455 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:11.860645 (+ 190us) server_negotiation.cc:299] Negotiation successful
0621 07:03:11.860711 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:03:11.862694 24594 raft_consensus.cc:348] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } }
I20260621 07:03:11.863161 24594 raft_consensus.cc:374] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:11.863270 24594 raft_consensus.cc:729] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c083f66936d14d78a9dee19cd2d2b40a, State: Initialized, Role: FOLLOWER
I20260621 07:03:11.863842 24594 consensus_queue.cc:260] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } }
I20260621 07:03:11.864084 24594 raft_consensus.cc:388] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:11.864228 24594 raft_consensus.cc:482] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:11.864396 24594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:11.867071 24594 raft_consensus.cc:504] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } }
I20260621 07:03:11.867631 24594 leader_election.cc:302] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c083f66936d14d78a9dee19cd2d2b40a; no voters:
I20260621 07:03:11.868202 24594 leader_election.cc:288] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:11.868423 24604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:11.869134 24604 raft_consensus.cc:686] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [term 1 LEADER]: Becoming Leader. State: Replica: c083f66936d14d78a9dee19cd2d2b40a, State: Running, Role: LEADER
I20260621 07:03:11.869674 24604 consensus_queue.cc:237] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } }
I20260621 07:03:11.870296 24594 sys_catalog.cc:564] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:11.872517 24605 sys_catalog.cc:455] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c083f66936d14d78a9dee19cd2d2b40a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } } }
I20260621 07:03:11.872664 24606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c083f66936d14d78a9dee19cd2d2b40a. Latest consensus state: current_term: 1 leader_uuid: "c083f66936d14d78a9dee19cd2d2b40a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c083f66936d14d78a9dee19cd2d2b40a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40999 } } }
I20260621 07:03:11.873006 24605 sys_catalog.cc:458] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:11.877734 24606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:11.878537 24612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:11.883667 24612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:11.892230 24612 catalog_manager.cc:1347] Generated new cluster ID: 75df2b572659417cb53ffa4ce7ce1767
I20260621 07:03:11.892393 24612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:11.899262 24612 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:11.901041 24612 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:11.915089 24612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c083f66936d14d78a9dee19cd2d2b40a: Generated new TSK 0
I20260621 07:03:11.916044 24612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:11.926630 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24527
W20260621 07:03:11.928181 24589 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.92:50651
W20260621 07:03:11.928978 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24527: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:11.929138 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24527
W20260621 07:03:11.946112 24512 connection.cc:535] client connection to 127.0.0.1:40999 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40999): Connection reset by peer (error 104)
2026-06-21T07:03:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1953 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:12 dist-test-slave-9zdv krb5kdc[24625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:12 dist-test-slave-9zdv krb5kdc[24625](info): set up 2 sockets
Jun 21 07:03:12 dist-test-slave-9zdv krb5kdc[24625](info): commencing operation
krb5kdc: starting...
W20260621 07:03:13.039652 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:13 dist-test-slave-9zdv krb5kdc[24625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 07:03:13.295974 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:13.631544 24638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:13.631842 24638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:13.631932 24638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:13.639113 24638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:13.639232 24638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:13.639278 24638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:13.639400 24638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:13.639475 24638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:13.649418 24638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:13.650955 24638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:13.652182 24638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:13.664036 24644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:13.664176 24643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:13.664093 24646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:13.664810 24638 server_base.cc:1034] running on GCE node
I20260621 07:03:13.665614 24638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:13.667168 24638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:13.668395 24638 hybrid_clock.cc:648] HybridClock initialized: now 1782025393668351 us; error 41 us; skew 500 ppm
I20260621 07:03:13.668782 24638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:13 dist-test-slave-9zdv krb5kdc[24625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:13.677537 24638 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 07:03:13.679092 24638 webserver.cc:460] Webserver started at http://127.0.0.1:40811/ using document root <none> and password file <none>
I20260621 07:03:13.679764 24638 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:13.679863 24638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:13.680122 24638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:13.683157 24638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "2ff4add65fae43609db76b2d0ba14de6"
format_stamp: "Formatted at 2026-06-21 07:03:13 on dist-test-slave-9zdv"
server_key: "ea34cda671db1b5d2f11cedcdf413dad"
server_key_iv: "22c833fcc8d747eb4c5c15102e4dc9dd"
server_key_version: "encryptionkey@0"
I20260621 07:03:13.683882 24638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "2ff4add65fae43609db76b2d0ba14de6"
format_stamp: "Formatted at 2026-06-21 07:03:13 on dist-test-slave-9zdv"
server_key: "ea34cda671db1b5d2f11cedcdf413dad"
server_key_iv: "22c833fcc8d747eb4c5c15102e4dc9dd"
server_key_version: "encryptionkey@0"
I20260621 07:03:13.689921 24638 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260621 07:03:13.695048 24653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:13.696532 24638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:03:13.696676 24638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "2ff4add65fae43609db76b2d0ba14de6"
format_stamp: "Formatted at 2026-06-21 07:03:13 on dist-test-slave-9zdv"
server_key: "ea34cda671db1b5d2f11cedcdf413dad"
server_key_iv: "22c833fcc8d747eb4c5c15102e4dc9dd"
server_key_version: "encryptionkey@0"
I20260621 07:03:13.696841 24638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:13.702454 24638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:13.706357 24638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:13.706591 24638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:13.725072 24638 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35579
I20260621 07:03:13.725112 24704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35579 every 8 connection(s)
I20260621 07:03:13.726500 24638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:13.730441 24705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:13.731312 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24638
I20260621 07:03:13.731504 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:13.732056 3031 external_mini_cluster.cc:1428] Setting key c01ee78c5bf13177053be4f6f56b1787
I20260621 07:03:13.745352 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:13.739528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:56109 (local address 127.0.0.1:35579)
0621 07:03:13.740311 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:13.740323 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:13.740365 (+ 42us) server_negotiation.cc:407] Connection header received
0621 07:03:13.741945 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:13.741967 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:13.742329 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:13.742696 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:13.743780 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:13.743787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:13.743802 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:13.743824 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:13.743877 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:13.743899 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:13.744090 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:13.744188 (+ 98us) server_negotiation.cc:299] Negotiation successful
0621 07:03:13.744393 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":369,"threads_started":1}
I20260621 07:03:13.746119 24705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6: Bootstrap starting.
I20260621 07:03:13.749387 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:13.750110 24705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:13.751412 24705 log.cc:824] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:13.754563 24705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6: No bootstrap required, opened a new log
I20260621 07:03:13.757018 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:13.754490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.92:55885 (local address 127.0.0.1:35579)
0621 07:03:13.754679 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:13.754685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:13.755078 (+ 393us) server_negotiation.cc:407] Connection header received
0621 07:03:13.755225 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:13.755231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:13.755288 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:13.755413 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:13.756323 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:13.756329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:13.756333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:13.756351 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:13.756398 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:13.756428 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:13.756550 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:13.756638 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 07:03:13.756701 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260621 07:03:13.763509 24705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260621 07:03:13.763943 24705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:13.764057 24705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff4add65fae43609db76b2d0ba14de6, State: Initialized, Role: FOLLOWER
I20260621 07:03:13.764803 24705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260621 07:03:13.765096 24705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:13.765256 24705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:13.765427 24705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:13.767706 24705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260621 07:03:13.768229 24705 leader_election.cc:302] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ff4add65fae43609db76b2d0ba14de6; no voters:
I20260621 07:03:13.768841 24705 leader_election.cc:288] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:13.769011 24715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:13.770124 24715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff4add65fae43609db76b2d0ba14de6, State: Running, Role: LEADER
I20260621 07:03:13.770740 24715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260621 07:03:13.771023 24705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:13.773259 24717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ff4add65fae43609db76b2d0ba14de6. Latest consensus state: current_term: 1 leader_uuid: "2ff4add65fae43609db76b2d0ba14de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } } }
I20260621 07:03:13.773623 24717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:13.773907 24716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ff4add65fae43609db76b2d0ba14de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff4add65fae43609db76b2d0ba14de6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } } }
I20260621 07:03:13.774222 24716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:13.775954 24722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:13.781359 24722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:13.789777 24722 catalog_manager.cc:1347] Generated new cluster ID: 4461bfa0f8a14aeca92718f73474f2bf
I20260621 07:03:13.789889 24722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:13.810472 24722 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:13.810837 24722 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:13.816164 24722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ff4add65fae43609db76b2d0ba14de6: Generated new TSK 0
I20260621 07:03:13.816896 24722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:13.822110 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24638
W20260621 07:03:13.823241 24700 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.92:56109
W20260621 07:03:13.823951 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24638: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:13.824100 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24638
2026-06-21T07:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1893 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:13 dist-test-slave-9zdv krb5kdc[24736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:13 dist-test-slave-9zdv krb5kdc[24736](info): set up 2 sockets
Jun 21 07:03:13 dist-test-slave-9zdv krb5kdc[24736](info): commencing operation
krb5kdc: starting...
W20260621 07:03:14.919785 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:15.178520 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35979
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:35979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:15.510721 24749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:15.511041 24749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:15.511238 24749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:15.518544 24749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:15.518687 24749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:15.518780 24749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:15.518893 24749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:15.518975 24749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:15.529232 24749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:35979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:35979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:15.530573 24749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:15.531952 24749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:15.543613 24754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:15.543677 24755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:15.543766 24757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:15.544780 24749 server_base.cc:1034] running on GCE node
I20260621 07:03:15.545497 24749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:15.546836 24749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:15.548065 24749 hybrid_clock.cc:648] HybridClock initialized: now 1782025395548051 us; error 68 us; skew 500 ppm
I20260621 07:03:15.548435 24749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025395, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:15.551539 24749 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:15.552996 24749 webserver.cc:460] Webserver started at http://127.2.245.254:44557/ using document root <none> and password file <none>
I20260621 07:03:15.553689 24749 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:15.553845 24749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:15.554157 24749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:15.557389 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "26b5d51f2a604709a2ce78bd6e294e18"
format_stamp: "Formatted at 2026-06-21 07:03:15 on dist-test-slave-9zdv"
server_key: "60d9d0be9316b9cd7f6a337c384ca4f2"
server_key_iv: "f236f200b1a6715a02607aa9674f4048"
server_key_version: "encryptionkey@0"
I20260621 07:03:15.558234 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "26b5d51f2a604709a2ce78bd6e294e18"
format_stamp: "Formatted at 2026-06-21 07:03:15 on dist-test-slave-9zdv"
server_key: "60d9d0be9316b9cd7f6a337c384ca4f2"
server_key_iv: "f236f200b1a6715a02607aa9674f4048"
server_key_version: "encryptionkey@0"
I20260621 07:03:15.563591 24749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:03:15.567090 24764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:15.568404 24749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:03:15.568599 24749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "26b5d51f2a604709a2ce78bd6e294e18"
format_stamp: "Formatted at 2026-06-21 07:03:15 on dist-test-slave-9zdv"
server_key: "60d9d0be9316b9cd7f6a337c384ca4f2"
server_key_iv: "f236f200b1a6715a02607aa9674f4048"
server_key_version: "encryptionkey@0"
I20260621 07:03:15.568800 24749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:15.595588 24749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:15.599910 24749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:15.600222 24749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:15.618868 24749 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:35979
I20260621 07:03:15.618886 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:35979 every 8 connection(s)
I20260621 07:03:15.620291 24749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:15.623984 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24749
I20260621 07:03:15.624173 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:15.624354 24816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:15.624745 3031 external_mini_cluster.cc:1428] Setting key 4af3fa94b93c93e75540195612668ed8
I20260621 07:03:15.639056 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18: Bootstrap starting.
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:15.643134 24816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:15.644547 24816 log.cc:824] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:15.648155 24816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18: No bootstrap required, opened a new log
I20260621 07:03:15.649861 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:15.626535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37511 (local address 127.2.245.254:35979)
0621 07:03:15.627364 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:15.627378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:15.627424 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:03:15.629080 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:15.629101 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:15.629521 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:15.629859 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:15.631655 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:15.632906 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:15.634667 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:15.635432 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:15.635618 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:15.640725 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:15.640738 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:15.640759 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:15.640815 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:15.644914 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:15.646084 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:15.646093 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:15.646103 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:15.646289 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:15.646729 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:15.646734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:15.646737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:15.647424 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:15.647733 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:15.648173 (+ 440us) server_negotiation.cc:299] Negotiation successful
0621 07:03:15.648498 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":397,"threads_started":1}
I20260621 07:03:15.653851 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:15.658072 24816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } }
I20260621 07:03:15.658550 24816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:15.658666 24816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b5d51f2a604709a2ce78bd6e294e18, State: Initialized, Role: FOLLOWER
I20260621 07:03:15.659412 24816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } }
I20260621 07:03:15.659734 24816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:15.659910 24816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:15.660097 24816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:15.663241 24816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } }
I20260621 07:03:15.663895 24816 leader_election.cc:302] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26b5d51f2a604709a2ce78bd6e294e18; no voters:
I20260621 07:03:15.664547 24816 leader_election.cc:288] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:15.664695 24826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:15.666265 24826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [term 1 LEADER]: Becoming Leader. State: Replica: 26b5d51f2a604709a2ce78bd6e294e18, State: Running, Role: LEADER
I20260621 07:03:15.667101 24816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:15.666980 24826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } }
I20260621 07:03:15.669711 24827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26b5d51f2a604709a2ce78bd6e294e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } } }
I20260621 07:03:15.670145 24827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:15.676551 24828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26b5d51f2a604709a2ce78bd6e294e18. Latest consensus state: current_term: 1 leader_uuid: "26b5d51f2a604709a2ce78bd6e294e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b5d51f2a604709a2ce78bd6e294e18" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 35979 } } }
I20260621 07:03:15.677184 24828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:15.677428 24831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:15.684101 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:15.659180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33687 (local address 127.2.245.254:35979)
0621 07:03:15.659395 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:15.659407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:15.660027 (+ 620us) server_negotiation.cc:407] Connection header received
0621 07:03:15.660251 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:15.660256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:15.660333 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:15.660413 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:15.662282 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:15.663143 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:15.664959 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:15.665717 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:15.665812 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:15.669850 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:15.669860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:15.669868 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:15.669928 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:15.678608 (+ 8680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:15.680006 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:15.680020 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:15.680026 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:15.680228 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:15.680867 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:15.680877 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:15.680883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:15.681491 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:15.681919 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:15.683488 (+ 1569us) server_negotiation.cc:299] Negotiation successful
0621 07:03:15.683686 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:03:15.686726 24831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:15.704594 24831 catalog_manager.cc:1347] Generated new cluster ID: 3f3270ba01d44836a86b572ad27a2f6e
I20260621 07:03:15.704715 24831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:15.722314 24831 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:15.723701 24831 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:15.732373 24831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26b5d51f2a604709a2ce78bd6e294e18: Generated new TSK 0
I20260621 07:03:15.733211 24831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:15.741671 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24749
I20260621 07:03:15.742868 24811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:15.833719 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24749
W20260621 07:03:15.850874 24734 connection.cc:535] client connection to 127.2.245.254:35979 recv error: Network error: failed to read from TLS socket (remote: 127.2.245.254:35979): Connection reset by peer (error 104)
2026-06-21T07:03:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2010 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24848](info): set up 2 sockets
Jun 21 07:03:15 dist-test-slave-9zdv krb5kdc[24848](info): commencing operation
krb5kdc: starting...
W20260621 07:03:16.935449 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:17 dist-test-slave-9zdv krb5kdc[24848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:17.190857 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36663
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:17.524205 24861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:17.524488 24861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:17.524657 24861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:17.531953 24861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:17.532068 24861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:17.532177 24861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:17.532279 24861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:17.532351 24861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:17.543578 24861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:36663
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:17.545300 24861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:17.547183 24861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:17.561544 24866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:17.561573 24867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:17.561874 24869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:17.562740 24861 server_base.cc:1034] running on GCE node
I20260621 07:03:17.563549 24861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:17.565052 24861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:17.566251 24861 hybrid_clock.cc:648] HybridClock initialized: now 1782025397566211 us; error 37 us; skew 500 ppm
I20260621 07:03:17.566607 24861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:17 dist-test-slave-9zdv krb5kdc[24848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025397, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:17.569798 24861 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:17.571357 24861 webserver.cc:460] Webserver started at http://127.2.245.254:33751/ using document root <none> and password file <none>
I20260621 07:03:17.572036 24861 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:17.572166 24861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:17.572458 24861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:17.575475 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "24ac462db10748b7b3398a533cb79e00"
format_stamp: "Formatted at 2026-06-21 07:03:17 on dist-test-slave-9zdv"
server_key: "9c00a219d91f5b1e411729f94f831e81"
server_key_iv: "5593e6b27c0426d75f49055adf632179"
server_key_version: "encryptionkey@0"
I20260621 07:03:17.576273 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "24ac462db10748b7b3398a533cb79e00"
format_stamp: "Formatted at 2026-06-21 07:03:17 on dist-test-slave-9zdv"
server_key: "9c00a219d91f5b1e411729f94f831e81"
server_key_iv: "5593e6b27c0426d75f49055adf632179"
server_key_version: "encryptionkey@0"
I20260621 07:03:17.581421 24861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:03:17.585069 24876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:17.586400 24861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:03:17.586611 24861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "24ac462db10748b7b3398a533cb79e00"
format_stamp: "Formatted at 2026-06-21 07:03:17 on dist-test-slave-9zdv"
server_key: "9c00a219d91f5b1e411729f94f831e81"
server_key_iv: "5593e6b27c0426d75f49055adf632179"
server_key_version: "encryptionkey@0"
I20260621 07:03:17.586804 24861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:17.613142 24861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:17.617393 24861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:17.617692 24861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:17.635622 24861 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:36663
I20260621 07:03:17.635643 24927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:36663 every 8 connection(s)
I20260621 07:03:17.636994 24861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:17.640823 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:17.646283 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24861
I20260621 07:03:17.646456 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:17.646868 3031 external_mini_cluster.cc:1428] Setting key b62a8833f33571346b3d03d365a934ab
I20260621 07:03:17.652866 24928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00: Bootstrap starting.
I20260621 07:03:17.656802 24928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:17.658051 24928 log.cc:824] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:17.661434 24928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00: No bootstrap required, opened a new log
I20260621 07:03:17.662287 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:17.648741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55119 (local address 127.2.245.254:36663)
0621 07:03:17.649690 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:17.649707 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:17.649757 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:03:17.652191 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:17.652218 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:17.652757 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:17.653210 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:17.653773 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:17.655888 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:17.657272 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:17.658322 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:17.658520 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:17.659882 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:17.659890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:17.659904 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:17.660445 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:17.660511 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:17.660544 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:17.660667 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:17.660996 (+ 329us) server_negotiation.cc:299] Negotiation successful
0621 07:03:17.661203 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":371,"threads_started":1}
I20260621 07:03:17.666463 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:17.670171 24928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } }
I20260621 07:03:17.670583 24928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:17.670712 24928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ac462db10748b7b3398a533cb79e00, State: Initialized, Role: FOLLOWER
I20260621 07:03:17.671366 24928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } }
I20260621 07:03:17.671625 24928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:17.671777 24928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:17.671958 24928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:17.674901 24928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } }
I20260621 07:03:17.675586 24928 leader_election.cc:302] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ac462db10748b7b3398a533cb79e00; no voters:
I20260621 07:03:17.676303 24928 leader_election.cc:288] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:17.676429 24938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:17.677954 24938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [term 1 LEADER]: Becoming Leader. State: Replica: 24ac462db10748b7b3398a533cb79e00, State: Running, Role: LEADER
I20260621 07:03:17.678774 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:17.670963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52665 (local address 127.2.245.254:36663)
0621 07:03:17.671134 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:17.671140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:17.671790 (+ 650us) server_negotiation.cc:407] Connection header received
0621 07:03:17.671996 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:17.672001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:17.672096 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:17.672212 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:17.672637 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:17.673789 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:17.675294 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:17.675955 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:17.676123 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:17.677602 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:17.677611 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:17.677618 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:17.677647 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:17.677734 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:17.677762 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:17.677894 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:17.678375 (+ 481us) server_negotiation.cc:299] Negotiation successful
0621 07:03:17.678428 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:03:17.678810 24928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:17.678822 24938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } }
I20260621 07:03:17.684809 24940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24ac462db10748b7b3398a533cb79e00. Latest consensus state: current_term: 1 leader_uuid: "24ac462db10748b7b3398a533cb79e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } } }
I20260621 07:03:17.685317 24940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:17.685274 24939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24ac462db10748b7b3398a533cb79e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ac462db10748b7b3398a533cb79e00" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 36663 } } }
I20260621 07:03:17.685550 24939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:17.686228 24951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:17.692982 24953 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:03:17.699198 24951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:17.707896 24951 catalog_manager.cc:1347] Generated new cluster ID: 5211513f241446d8837a4016bc6b9263
I20260621 07:03:17.708137 24951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:17.749064 24951 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:17.750910 24951 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:17.765575 24951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24ac462db10748b7b3398a533cb79e00: Generated new TSK 0
I20260621 07:03:17.766450 24951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:17.780789 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24861
W20260621 07:03:17.782194 24923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55119
W20260621 07:03:17.783176 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24861: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:17.783334 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24861
W20260621 07:03:17.800762 24846 connection.cc:535] client connection to 127.2.245.254:36663 recv error: Network error: failed to read from TLS socket (remote: 127.2.245.254:36663): Connection reset by peer (error 104)
2026-06-21T07:03:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1950 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:17 dist-test-slave-9zdv krb5kdc[24959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:17 dist-test-slave-9zdv krb5kdc[24959](info): set up 2 sockets
Jun 21 07:03:17 dist-test-slave-9zdv krb5kdc[24959](info): commencing operation
krb5kdc: starting...
W20260621 07:03:18.887645 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:19 dist-test-slave-9zdv krb5kdc[24959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:19.144395 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33989
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:34733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:33989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:19.479882 24972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:19.480166 24972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:19.480257 24972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:19.487540 24972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:19.487664 24972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:19.487743 24972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:19.487833 24972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:19.487893 24972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:19.497709 24972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:34733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:33989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:19.499236 24972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:19.500684 24972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:19.513504 24977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:19.513571 24978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:19.513531 24980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:19.514250 24972 server_base.cc:1034] running on GCE node
I20260621 07:03:19.515188 24972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:19.516768 24972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:19.518014 24972 hybrid_clock.cc:648] HybridClock initialized: now 1782025399517979 us; error 55 us; skew 500 ppm
I20260621 07:03:19.518395 24972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:19 dist-test-slave-9zdv krb5kdc[24959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025399, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:19.525946 24972 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:19.527669 24972 webserver.cc:460] Webserver started at http://127.2.245.254:35001/ using document root <none> and password file <none>
I20260621 07:03:19.528386 24972 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:19.528537 24972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:19.528949 24972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:19.532226 24972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "73ba38732434419292da5fddbc7175d4"
format_stamp: "Formatted at 2026-06-21 07:03:19 on dist-test-slave-9zdv"
server_key: "57de5ed77b295a52604fb68bc9db5418"
server_key_iv: "bc46b99c6cfe07b8e853d5b0cc404873"
server_key_version: "encryptionkey@0"
I20260621 07:03:19.533138 24972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "73ba38732434419292da5fddbc7175d4"
format_stamp: "Formatted at 2026-06-21 07:03:19 on dist-test-slave-9zdv"
server_key: "57de5ed77b295a52604fb68bc9db5418"
server_key_iv: "bc46b99c6cfe07b8e853d5b0cc404873"
server_key_version: "encryptionkey@0"
I20260621 07:03:19.538758 24972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260621 07:03:19.542711 24987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:19.544297 24972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:03:19.544512 24972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "73ba38732434419292da5fddbc7175d4"
format_stamp: "Formatted at 2026-06-21 07:03:19 on dist-test-slave-9zdv"
server_key: "57de5ed77b295a52604fb68bc9db5418"
server_key_iv: "bc46b99c6cfe07b8e853d5b0cc404873"
server_key_version: "encryptionkey@0"
I20260621 07:03:19.544782 24972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:19.551399 24972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:19.555948 24972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:19.556293 24972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:19.576588 24972 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:33989
I20260621 07:03:19.576601 25038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:33989 every 8 connection(s)
I20260621 07:03:19.578142 24972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:19.579723 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 24972
I20260621 07:03:19.579901 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:19.580440 3031 external_mini_cluster.cc:1428] Setting key 7df474fd510370784a659ca1e3f17e32
I20260621 07:03:19.583071 25039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:19.588804 25042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:19.582353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50893 (local address 127.2.245.254:33989)
0621 07:03:19.583303 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:19.583315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:19.583363 (+ 48us) server_negotiation.cc:407] Connection header received
0621 07:03:19.585154 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:19.585177 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:19.585544 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:19.586035 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:19.587164 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:19.587170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:19.587183 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:19.587200 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:19.587274 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:19.587293 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:19.587456 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:19.587604 (+ 148us) server_negotiation.cc:299] Negotiation successful
0621 07:03:19.587840 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":414,"threads_started":1}
I20260621 07:03:19.592514 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:19.596517 25039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4: Bootstrap starting.
I20260621 07:03:19.600464 25039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:19.600857 25042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:19.597904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40353 (local address 127.2.245.254:33989)
0621 07:03:19.598127 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:19.598133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:19.598628 (+ 495us) server_negotiation.cc:407] Connection header received
0621 07:03:19.598769 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:19.598774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:19.598832 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:19.598968 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:19.600175 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:19.600181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:19.600186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:19.600205 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:19.600253 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:19.600270 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:19.600397 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:19.600518 (+ 121us) server_negotiation.cc:299] Negotiation successful
0621 07:03:19.600581 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:03:19.601760 25039 log.cc:824] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:19.604486 25039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4: No bootstrap required, opened a new log
I20260621 07:03:19.612339 25039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } }
I20260621 07:03:19.612764 25039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:19.612875 25039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ba38732434419292da5fddbc7175d4, State: Initialized, Role: FOLLOWER
I20260621 07:03:19.613473 25039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } }
I20260621 07:03:19.613718 25039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:19.613868 25039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:19.614034 25039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:19.616360 25039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } }
I20260621 07:03:19.616856 25039 leader_election.cc:302] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73ba38732434419292da5fddbc7175d4; no voters:
I20260621 07:03:19.617468 25039 leader_election.cc:288] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:19.617654 25049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:19.618342 25049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [term 1 LEADER]: Becoming Leader. State: Replica: 73ba38732434419292da5fddbc7175d4, State: Running, Role: LEADER
I20260621 07:03:19.618891 25049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } }
I20260621 07:03:19.619655 25039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:19.621543 25050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73ba38732434419292da5fddbc7175d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } } }
I20260621 07:03:19.621966 25050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:19.622310 25051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73ba38732434419292da5fddbc7175d4. Latest consensus state: current_term: 1 leader_uuid: "73ba38732434419292da5fddbc7175d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ba38732434419292da5fddbc7175d4" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33989 } } }
I20260621 07:03:19.622655 25051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:19.624802 25054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:19.630200 25054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:19.639390 25054 catalog_manager.cc:1347] Generated new cluster ID: 191fbac4812841228ed4a60c23807e34
I20260621 07:03:19.639508 25054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:19.654048 25054 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:19.654484 25054 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:19.660112 25054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73ba38732434419292da5fddbc7175d4: Generated new TSK 0
I20260621 07:03:19.660902 25054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:19.673703 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24972
W20260621 07:03:19.675328 25034 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50893
W20260621 07:03:19.676033 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 24972: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:19.676198 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 24972
2026-06-21T07:03:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1894 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:19 dist-test-slave-9zdv krb5kdc[25070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:19 dist-test-slave-9zdv krb5kdc[25070](info): set up 2 sockets
Jun 21 07:03:19 dist-test-slave-9zdv krb5kdc[25070](info): commencing operation
krb5kdc: starting...
W20260621 07:03:20.799728 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:20 dist-test-slave-9zdv krb5kdc[25070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:21.064248 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:40489
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:40489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:21.398851 25083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:21.399291 25083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:21.399446 25083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:21.406810 25083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:21.406999 25083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:21.407109 25083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:21.407218 25083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:21.407301 25083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:21.417552 25083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:40489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:40489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:21.419030 25083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:21.420382 25083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:21.432384 25088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:21.432340 25091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:21.432428 25089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:21.433650 25083 server_base.cc:1034] running on GCE node
I20260621 07:03:21.434509 25083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:21.436044 25083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:21.437258 25083 hybrid_clock.cc:648] HybridClock initialized: now 1782025401437220 us; error 32 us; skew 500 ppm
I20260621 07:03:21.437624 25083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:21 dist-test-slave-9zdv krb5kdc[25070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:21.440786 25083 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:21.442243 25083 webserver.cc:460] Webserver started at http://127.2.245.254:44329/ using document root <none> and password file <none>
I20260621 07:03:21.442896 25083 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:21.443059 25083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:21.443331 25083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:21.446261 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "0401fa4ddca24be4bcad7b7446d5e764"
format_stamp: "Formatted at 2026-06-21 07:03:21 on dist-test-slave-9zdv"
server_key: "7b4619b0d72478b27548645a343b74b1"
server_key_iv: "e32b61eba09054a584515f77d2d1632b"
server_key_version: "encryptionkey@0"
I20260621 07:03:21.447036 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "0401fa4ddca24be4bcad7b7446d5e764"
format_stamp: "Formatted at 2026-06-21 07:03:21 on dist-test-slave-9zdv"
server_key: "7b4619b0d72478b27548645a343b74b1"
server_key_iv: "e32b61eba09054a584515f77d2d1632b"
server_key_version: "encryptionkey@0"
I20260621 07:03:21.452256 25083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260621 07:03:21.455785 25098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:21.457136 25083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260621 07:03:21.457286 25083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "0401fa4ddca24be4bcad7b7446d5e764"
format_stamp: "Formatted at 2026-06-21 07:03:21 on dist-test-slave-9zdv"
server_key: "7b4619b0d72478b27548645a343b74b1"
server_key_iv: "e32b61eba09054a584515f77d2d1632b"
server_key_version: "encryptionkey@0"
I20260621 07:03:21.457458 25083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:21.484850 25083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:21.489053 25083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:21.489310 25083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:21.508167 25083 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:40489
I20260621 07:03:21.508178 25149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:40489 every 8 connection(s)
I20260621 07:03:21.509617 25083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:21.510282 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25083
I20260621 07:03:21.510476 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:21.511122 3031 external_mini_cluster.cc:1428] Setting key 516c339afd0e52985f624e701e115e9b
I20260621 07:03:21.514567 25150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 21 07:03:21 dist-test-slave-9zdv krb5kdc[25070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:21.528498 25150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: Bootstrap starting.
I20260621 07:03:21.532758 25150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:21.534137 25150 log.cc:824] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:21.537082 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:21.513051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42429 (local address 127.2.245.254:40489)
0621 07:03:21.513964 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:21.513979 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:21.514030 (+ 51us) server_negotiation.cc:407] Connection header received
0621 07:03:21.516388 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:21.516416 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:21.516932 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:21.517388 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:21.518721 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:21.520871 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:21.522161 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:21.523245 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:21.523427 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:21.528185 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:21.528198 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:21.528219 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:21.528288 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:21.532268 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:21.533114 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:21.533122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:21.533144 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:21.533372 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:21.533760 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:21.533766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:21.533769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:21.534510 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:21.534850 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:21.535335 (+ 485us) server_negotiation.cc:299] Negotiation successful
0621 07:03:21.535747 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":336,"threads_started":1}
I20260621 07:03:21.537719 25150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: No bootstrap required, opened a new log
I20260621 07:03:21.540988 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:21.546208 25150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } }
I20260621 07:03:21.546706 25150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:21.546819 25150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0401fa4ddca24be4bcad7b7446d5e764, State: Initialized, Role: FOLLOWER
I20260621 07:03:21.547585 25150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } }
I20260621 07:03:21.547866 25150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:21.548019 25150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:21.548206 25150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:21.551378 25150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } }
I20260621 07:03:21.552000 25150 leader_election.cc:302] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0401fa4ddca24be4bcad7b7446d5e764; no voters:
I20260621 07:03:21.552762 25150 leader_election.cc:288] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:21.552876 25160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:21.555230 25160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [term 1 LEADER]: Becoming Leader. State: Replica: 0401fa4ddca24be4bcad7b7446d5e764, State: Running, Role: LEADER
I20260621 07:03:21.555924 25160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } }
I20260621 07:03:21.555310 25150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:21.560957 25161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0401fa4ddca24be4bcad7b7446d5e764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } } }
I20260621 07:03:21.561424 25161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:21.562081 25162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0401fa4ddca24be4bcad7b7446d5e764. Latest consensus state: current_term: 1 leader_uuid: "0401fa4ddca24be4bcad7b7446d5e764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0401fa4ddca24be4bcad7b7446d5e764" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 40489 } } }
I20260621 07:03:21.562355 25162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:21.563844 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:21.545381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35881 (local address 127.2.245.254:40489)
0621 07:03:21.545612 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:21.545620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:21.545998 (+ 378us) server_negotiation.cc:407] Connection header received
0621 07:03:21.546159 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:21.546164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:21.546228 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:21.546305 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:21.548227 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:21.549178 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:21.550998 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:21.551477 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:21.551626 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:21.554701 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:21.554711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:21.554718 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:21.554801 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:21.558658 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:21.559555 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:21.559560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:21.559564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:21.559760 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:21.561900 (+ 2140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:21.561905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:21.561908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:21.562377 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:21.562683 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:21.563113 (+ 430us) server_negotiation.cc:299] Negotiation successful
0621 07:03:21.563353 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":67}
W20260621 07:03:21.569926 25176 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:03:21.570050 25176 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:03:21.570230 25170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:21.572458 25170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:21.581151 25170 catalog_manager.cc:1347] Generated new cluster ID: 0886eec68f12463f9a88a534ec10ad20
I20260621 07:03:21.581269 25170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:21.600844 25170 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:21.602125 25170 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:21.613071 25170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0401fa4ddca24be4bcad7b7446d5e764: Generated new TSK 0
I20260621 07:03:21.613979 25170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:21.619967 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25083
I20260621 07:03:21.621081 25145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:21.716137 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25083
2026-06-21T07:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2037 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:21 dist-test-slave-9zdv krb5kdc[25182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:21 dist-test-slave-9zdv krb5kdc[25182](info): set up 2 sockets
Jun 21 07:03:21 dist-test-slave-9zdv krb5kdc[25182](info): commencing operation
krb5kdc: starting...
W20260621 07:03:22.819550 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:22 dist-test-slave-9zdv krb5kdc[25182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:23.082005 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33483
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:33483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:23.415515 25195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:23.415822 25195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:23.416008 25195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:23.423414 25195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:23.423538 25195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:23.423661 25195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:23.423764 25195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:23.423830 25195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:23.433774 25195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:33483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:23.435185 25195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:23.436566 25195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:23.448603 25201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:23.448674 25200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:23.448683 25203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:23.449448 25195 server_base.cc:1034] running on GCE node
I20260621 07:03:23.450291 25195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:23.451865 25195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:23.453111 25195 hybrid_clock.cc:648] HybridClock initialized: now 1782025403453081 us; error 60 us; skew 500 ppm
I20260621 07:03:23.453467 25195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:23 dist-test-slave-9zdv krb5kdc[25182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025403, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:23.456646 25195 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:23.458148 25195 webserver.cc:460] Webserver started at http://127.2.245.254:40403/ using document root <none> and password file <none>
I20260621 07:03:23.458837 25195 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:23.459040 25195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:23.459368 25195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:23.462317 25195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "ba814deed50f4063b9b5d76154a9e018"
format_stamp: "Formatted at 2026-06-21 07:03:23 on dist-test-slave-9zdv"
server_key: "002e06d7bf1a6d2b64639ddc95901de3"
server_key_iv: "00d508bb864da797e4bee1d7f4f90259"
server_key_version: "encryptionkey@0"
I20260621 07:03:23.463167 25195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "ba814deed50f4063b9b5d76154a9e018"
format_stamp: "Formatted at 2026-06-21 07:03:23 on dist-test-slave-9zdv"
server_key: "002e06d7bf1a6d2b64639ddc95901de3"
server_key_iv: "00d508bb864da797e4bee1d7f4f90259"
server_key_version: "encryptionkey@0"
I20260621 07:03:23.468330 25195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260621 07:03:23.471872 25210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:23.473348 25195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:03:23.473519 25195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "ba814deed50f4063b9b5d76154a9e018"
format_stamp: "Formatted at 2026-06-21 07:03:23 on dist-test-slave-9zdv"
server_key: "002e06d7bf1a6d2b64639ddc95901de3"
server_key_iv: "00d508bb864da797e4bee1d7f4f90259"
server_key_version: "encryptionkey@0"
I20260621 07:03:23.473742 25195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:23.504904 25195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:23.509197 25195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:23.509502 25195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:23.528431 25195 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:33483
I20260621 07:03:23.528452 25261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:33483 every 8 connection(s)
I20260621 07:03:23.529901 25195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:23.533684 25262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:23.538249 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25195
I20260621 07:03:23.538429 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:23.539001 3031 external_mini_cluster.cc:1428] Setting key 2a042cfd953047014e49b7f6bfba37c9
I20260621 07:03:23.545365 25262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018: Bootstrap starting.
I20260621 07:03:23.548503 25262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:23.549551 25262 log.cc:824] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:23.552825 25262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018: No bootstrap required, opened a new log
I20260621 07:03:23.553066 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:23.540908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44279 (local address 127.2.245.254:33483)
0621 07:03:23.541755 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:23.541767 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:23.541811 (+ 44us) server_negotiation.cc:407] Connection header received
0621 07:03:23.544129 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:23.544153 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:23.544589 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:23.544991 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:23.545551 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:23.546834 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:23.548208 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:23.548998 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:23.549214 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:23.550507 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:23.550515 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:23.550528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:23.551218 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:23.551323 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:23.551359 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:23.551484 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:23.551802 (+ 318us) server_negotiation.cc:299] Negotiation successful
0621 07:03:23.552053 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":389,"threads_started":1}
I20260621 07:03:23.557144 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:23.561174 25262 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } }
I20260621 07:03:23.561771 25262 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:23.561908 25262 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba814deed50f4063b9b5d76154a9e018, State: Initialized, Role: FOLLOWER
I20260621 07:03:23.562556 25262 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } }
I20260621 07:03:23.562886 25262 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:23.563074 25262 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:23.563243 25262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:23.566037 25262 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } }
I20260621 07:03:23.566619 25262 leader_election.cc:302] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba814deed50f4063b9b5d76154a9e018; no voters:
I20260621 07:03:23.567364 25262 leader_election.cc:288] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:23.567502 25272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:23.568209 25272 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [term 1 LEADER]: Becoming Leader. State: Replica: ba814deed50f4063b9b5d76154a9e018, State: Running, Role: LEADER
I20260621 07:03:23.568866 25272 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } }
I20260621 07:03:23.569501 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:23.561544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44813 (local address 127.2.245.254:33483)
0621 07:03:23.561710 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:23.561717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:23.562306 (+ 589us) server_negotiation.cc:407] Connection header received
0621 07:03:23.562501 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:23.562507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:23.562602 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:23.562724 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:23.563345 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:23.564543 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:23.565819 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:23.566483 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:23.566615 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:23.567853 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:23.567862 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:23.567870 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:23.567902 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:23.567962 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:23.567993 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:23.568797 (+ 804us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:23.568864 (+ 67us) server_negotiation.cc:299] Negotiation successful
0621 07:03:23.568993 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":51}
I20260621 07:03:23.571718 25262 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:23.571772 25273 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba814deed50f4063b9b5d76154a9e018. Latest consensus state: current_term: 1 leader_uuid: "ba814deed50f4063b9b5d76154a9e018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } } }
I20260621 07:03:23.572237 25273 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:23.572093 25274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba814deed50f4063b9b5d76154a9e018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba814deed50f4063b9b5d76154a9e018" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 33483 } } }
I20260621 07:03:23.572361 25274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018 [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:23.573902 25276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:23.579461 25276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:23.589468 25276 catalog_manager.cc:1347] Generated new cluster ID: 72334a71d4e744dda40f649d6774b773
I20260621 07:03:23.589601 25276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:23.606451 25276 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:23.607769 25276 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:23.615837 25276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba814deed50f4063b9b5d76154a9e018: Generated new TSK 0
I20260621 07:03:23.616961 25276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:23.623889 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25195
W20260621 07:03:23.625053 25257 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44279
W20260621 07:03:23.625741 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25195: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:23.625901 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25195
2026-06-21T07:03:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1909 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:23 dist-test-slave-9zdv krb5kdc[25293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:23 dist-test-slave-9zdv krb5kdc[25293](info): set up 2 sockets
Jun 21 07:03:23 dist-test-slave-9zdv krb5kdc[25293](info): commencing operation
krb5kdc: starting...
W20260621 07:03:24.743805 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.010s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:24 dist-test-slave-9zdv krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:25.006098 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43493
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:25.337273 25306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:25.337638 25306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:25.337819 25306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:25.345217 25306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:25.345410 25306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:25.345523 25306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:25.345638 25306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:25.345716 25306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:25.355963 25306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:25.357407 25306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:25.358788 25306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:25.371016 25311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:25.370906 25312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:25.370981 25314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:25.371923 25306 server_base.cc:1034] running on GCE node
I20260621 07:03:25.372767 25306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:25.374240 25306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:25.375478 25306 hybrid_clock.cc:648] HybridClock initialized: now 1782025405375436 us; error 57 us; skew 500 ppm
I20260621 07:03:25.375840 25306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:25 dist-test-slave-9zdv krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025405, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:25.382359 25306 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:25.383947 25306 webserver.cc:460] Webserver started at http://127.2.245.254:45669/ using document root <none> and password file <none>
I20260621 07:03:25.384651 25306 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:25.384805 25306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:25.385116 25306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:25.388248 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "6f7aa168cb114bffb6998184a135433d"
format_stamp: "Formatted at 2026-06-21 07:03:25 on dist-test-slave-9zdv"
server_key: "2f4d2f6e504fcf39912eb517f4110c1c"
server_key_iv: "0651a6615cd07d55462ac32f75b626e3"
server_key_version: "encryptionkey@0"
I20260621 07:03:25.389065 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "6f7aa168cb114bffb6998184a135433d"
format_stamp: "Formatted at 2026-06-21 07:03:25 on dist-test-slave-9zdv"
server_key: "2f4d2f6e504fcf39912eb517f4110c1c"
server_key_iv: "0651a6615cd07d55462ac32f75b626e3"
server_key_version: "encryptionkey@0"
I20260621 07:03:25.394272 25306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260621 07:03:25.397811 25321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:25.399199 25306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260621 07:03:25.399394 25306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "6f7aa168cb114bffb6998184a135433d"
format_stamp: "Formatted at 2026-06-21 07:03:25 on dist-test-slave-9zdv"
server_key: "2f4d2f6e504fcf39912eb517f4110c1c"
server_key_iv: "0651a6615cd07d55462ac32f75b626e3"
server_key_version: "encryptionkey@0"
I20260621 07:03:25.399601 25306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:25.405329 25306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:25.409283 25306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:25.409586 25306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:25.428377 25306 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43493
I20260621 07:03:25.428401 25372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43493 every 8 connection(s)
I20260621 07:03:25.429842 25306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:25.431602 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25306
I20260621 07:03:25.431798 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:25.432226 3031 external_mini_cluster.cc:1428] Setting key 056705447a65e513bb049f3dde3b2636
I20260621 07:03:25.434186 25373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:25.441318 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:25.434133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46515 (local address 127.2.245.254:43493)
0621 07:03:25.435008 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:25.435023 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:25.435074 (+ 51us) server_negotiation.cc:407] Connection header received
0621 07:03:25.437253 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:25.437281 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:25.437765 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:25.438233 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:25.439285 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:25.439293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:25.439310 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:25.439328 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:25.439422 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:25.439447 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:25.439684 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:25.439836 (+ 152us) server_negotiation.cc:299] Negotiation successful
0621 07:03:25.440035 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":343,"threads_started":1}
I20260621 07:03:25.445062 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:25.447630 25373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d: Bootstrap starting.
I20260621 07:03:25.451601 25373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:25.452169 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:25.449378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33591 (local address 127.2.245.254:43493)
0621 07:03:25.449563 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:25.449571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:25.449941 (+ 370us) server_negotiation.cc:407] Connection header received
0621 07:03:25.450138 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:25.450142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:25.450190 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:25.450298 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:25.451441 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:25.451445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:25.451481 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:03:25.451503 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:25.451553 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:03:25.451594 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:25.451708 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:25.451827 (+ 119us) server_negotiation.cc:299] Negotiation successful
0621 07:03:25.451887 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:03:25.452953 25373 log.cc:824] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:25.455791 25373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d: No bootstrap required, opened a new log
I20260621 07:03:25.463166 25373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } }
I20260621 07:03:25.463572 25373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:03:25.463687 25373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7aa168cb114bffb6998184a135433d, State: Initialized, Role: FOLLOWER
I20260621 07:03:25.464263 25373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } }
I20260621 07:03:25.464511 25373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:03:25.464658 25373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:03:25.464833 25373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:25.467056 25373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } }
I20260621 07:03:25.467585 25373 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f7aa168cb114bffb6998184a135433d; no voters:
I20260621 07:03:25.468253 25373 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:03:25.468385 25383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:25.469192 25383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [term 1 LEADER]: Becoming Leader. State: Replica: 6f7aa168cb114bffb6998184a135433d, State: Running, Role: LEADER
I20260621 07:03:25.469775 25383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } }
I20260621 07:03:25.470454 25373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:25.472539 25384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f7aa168cb114bffb6998184a135433d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } } }
I20260621 07:03:25.472882 25384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:25.473780 25390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:25.473603 25385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7aa168cb114bffb6998184a135433d. Latest consensus state: current_term: 1 leader_uuid: "6f7aa168cb114bffb6998184a135433d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f7aa168cb114bffb6998184a135433d" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43493 } } }
I20260621 07:03:25.474020 25385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:25.480201 25390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:25.480902 25400 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:03:25.488766 25390 catalog_manager.cc:1347] Generated new cluster ID: aa97773482ff445a99e745394620cee1
I20260621 07:03:25.488884 25390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:25.502875 25390 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:25.503424 25390 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:25.515419 25390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f7aa168cb114bffb6998184a135433d: Generated new TSK 0
I20260621 07:03:25.516327 25390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:25.529510 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25306
W20260621 07:03:25.530804 25368 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46515
W20260621 07:03:25.531563 3031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25306: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260621 07:03:25.531729 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25306
2026-06-21T07:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1906 ms)
[----------] 18 tests from AuthTokenIssuingTest (35190 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:25 dist-test-slave-9zdv krb5kdc[25404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:25 dist-test-slave-9zdv krb5kdc[25404](info): set up 2 sockets
Jun 21 07:03:25 dist-test-slave-9zdv krb5kdc[25404](info): commencing operation
krb5kdc: starting...
W20260621 07:03:26.635870 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:26 dist-test-slave-9zdv krb5kdc[25404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:26.894305 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44003
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:27.231120 25417 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:27.231462 25417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:27.231528 25417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:27.231611 25417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:27.238842 25417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:27.238930 25417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:27.239053 25417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:27.239104 25417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:27.239158 25417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:27.249058 25417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:27.250366 25417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:27.251609 25417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:27.263428 25425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:27.263514 25422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:27.263486 25423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:27.264331 25417 server_base.cc:1034] running on GCE node
I20260621 07:03:27.265085 25417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:27.266562 25417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:27.267831 25417 hybrid_clock.cc:648] HybridClock initialized: now 1782025407267770 us; error 37 us; skew 500 ppm
I20260621 07:03:27.268167 25417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:27 dist-test-slave-9zdv krb5kdc[25404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:27.271452 25417 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:27.272954 25417 webserver.cc:460] Webserver started at http://127.2.245.254:33987/ using document root <none> and password file <none>
I20260621 07:03:27.273669 25417 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:27.273815 25417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:27.274145 25417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:27.277189 25417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "f0cc3f9c4d774925960cf58706571d0a"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "a0355311a13af4b0f2f68b135fb72e7a"
server_key_iv: "39c2db186c1d36898ef2e8c33f327e68"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.277976 25417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "f0cc3f9c4d774925960cf58706571d0a"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "a0355311a13af4b0f2f68b135fb72e7a"
server_key_iv: "39c2db186c1d36898ef2e8c33f327e68"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.283300 25417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:03:27.286809 25432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:27.288242 25417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:03:27.288424 25417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "f0cc3f9c4d774925960cf58706571d0a"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "a0355311a13af4b0f2f68b135fb72e7a"
server_key_iv: "39c2db186c1d36898ef2e8c33f327e68"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.288650 25417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:27.306087 25417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:27.310323 25417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:27.310604 25417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:27.329702 25417 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:44003
I20260621 07:03:27.329679 25483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:44003 every 8 connection(s)
I20260621 07:03:27.331058 25417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:27.334770 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:27.340026 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25417
I20260621 07:03:27.340263 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:27.340899 3031 external_mini_cluster.cc:1428] Setting key 8a1f793b8b10de9ad8dca139759d0450
I20260621 07:03:27.341334 25484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:27 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:27.365895 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.345094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44003 (local address 127.0.0.1:51062)
0621 07:03:27.346141 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.346203 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:27.348129 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:27.349140 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:27.349149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:27.349743 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:27.350109 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.350126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.351948 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.351952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.353045 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.353058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.354401 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.354406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.354529 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.355253 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:27.355275 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:27.357404 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:27.361734 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.361743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.361753 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.362152 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.362626 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.362629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.362631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.362830 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.363888 (+ 1058us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:27.363898 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:27.364142 (+ 244us) client_negotiation.cc:769] Sending connection context
0621 07:03:27.364363 (+ 221us) client_negotiation.cc:240] Negotiation successful
0621 07:03:27.364748 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":429,"threads_started":1}
I20260621 07:03:27.365921 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.345081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51062 (local address 127.2.245.254:44003)
0621 07:03:27.346795 (+ 1714us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:27.346808 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:27.346858 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:03:27.348513 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:27.348524 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:27.348842 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:27.349060 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:27.350331 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.351766 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.353196 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.354177 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.354372 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.357634 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:27.357646 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:27.357664 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:27.357708 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:27.361601 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.362286 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.362297 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.362304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.362503 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.362968 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.362975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.362981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.363686 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:27.363915 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:27.364545 (+ 630us) server_negotiation.cc:299] Negotiation successful
0621 07:03:27.364964 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":1490,"spinlock_wait_cycles":53632,"thread_start_us":1273,"threads_started":1}
WARNING: no policy specified for kudu/127.2.245.253@KRBTEST.COM; defaulting to no policy
I20260621 07:03:27.369292 25484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:27.370339 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.369948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44794)
0621 07:03:27.370122 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.370230 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260621 07:03:27.370599 25434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.253@KRBTEST.COM" created.
I20260621 07:03:27.372275 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.371935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44796)
0621 07:03:27.372081 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.372146 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260621 07:03:27.372493 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:39723: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
I20260621 07:03:27.402025 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:39723
--webserver_interface=127.2.245.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:03:27.421886 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } attempt: 1
I20260621 07:03:27.423269 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.422583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44804)
0621 07:03:27.422868 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.423057 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260621 07:03:27.424490 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.424007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44816)
0621 07:03:27.424200 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.424297 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260621 07:03:27.424716 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:39723: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
I20260621 07:03:27.493086 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } attempt: 2
I20260621 07:03:27.494590 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.494042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44832)
0621 07:03:27.494308 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.494418 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260621 07:03:27.495748 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.495312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44848)
0621 07:03:27.495493 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.495590 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260621 07:03:27.495952 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:39723: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
I20260621 07:03:27.587188 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } attempt: 3
I20260621 07:03:27.588491 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.587857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44856)
0621 07:03:27.588135 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.588283 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260621 07:03:27.589742 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.589239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44872)
0621 07:03:27.589424 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.589536 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 07:03:27.589954 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:39723: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
I20260621 07:03:27.733213 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } attempt: 4
I20260621 07:03:27.734513 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.733923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44876)
0621 07:03:27.734233 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.734346 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260621 07:03:27.735618 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.735210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44890)
0621 07:03:27.735374 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.735466 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260621 07:03:27.735821 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:39723: Network error: Client connection negotiation failed: client connection to 127.2.245.253:39723: connect: Connection refused (error 111)
W20260621 07:03:27.740518 25491 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:27.740861 25491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:27.740972 25491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:27.741106 25491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:27.748438 25491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:27.748517 25491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:27.748618 25491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:27.748695 25491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:27.748804 25491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:27.758611 25491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.245.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:27.759959 25491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:27.761257 25491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:27.773029 25496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:27.773070 25497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:27.773623 25499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:27.773927 25491 server_base.cc:1034] running on GCE node
I20260621 07:03:27.774763 25491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:27.776253 25491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:27.777494 25491 hybrid_clock.cc:648] HybridClock initialized: now 1782025407777431 us; error 36 us; skew 500 ppm
I20260621 07:03:27.777838 25491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:27 dist-test-slave-9zdv krb5kdc[25404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:27.781047 25491 init.cc:377] Logged in from keytab as kudu/127.2.245.253@KRBTEST.COM (short username kudu)
I20260621 07:03:27.782546 25491 webserver.cc:460] Webserver started at http://127.2.245.253:38905/ using document root <none> and password file <none>
I20260621 07:03:27.783339 25491 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:27.783469 25491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:27.783794 25491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:27.786784 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data/instance:
uuid: "265796c64ef8441cba127e7470656cfc"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "1ac487cd46345281321cbb41b0a5f676"
server_key_iv: "68a62f346ce33c579e213944e58b716c"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.787774 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal/instance:
uuid: "265796c64ef8441cba127e7470656cfc"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "1ac487cd46345281321cbb41b0a5f676"
server_key_iv: "68a62f346ce33c579e213944e58b716c"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.793220 25491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260621 07:03:27.796728 25506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:27.798097 25491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 07:03:27.798283 25491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "265796c64ef8441cba127e7470656cfc"
format_stamp: "Formatted at 2026-06-21 07:03:27 on dist-test-slave-9zdv"
server_key: "1ac487cd46345281321cbb41b0a5f676"
server_key_iv: "68a62f346ce33c579e213944e58b716c"
server_key_version: "encryptionkey@0"
I20260621 07:03:27.798513 25491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:27.825235 25491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:27.829487 25491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:27.829782 25491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:27.848568 25491 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.253:39723
I20260621 07:03:27.848575 25557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.253:39723 every 8 connection(s)
I20260621 07:03:27.849944 25491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
I20260621 07:03:27.853631 25558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:27.857363 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25491
I20260621 07:03:27.857563 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-1/wal/instance
I20260621 07:03:27.858080 3031 external_mini_cluster.cc:1428] Setting key 30eeade76c1e78ab1836916b9a8fdc5c
I20260621 07:03:27.862191 25558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:27 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.252@KRBTEST.COM; defaulting to no policy
I20260621 07:03:27.886329 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.865803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.2.245.254:44003)
0621 07:03:27.866021 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:27.866044 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:27.867350 (+ 1306us) server_negotiation.cc:407] Connection header received
0621 07:03:27.870031 (+ 2681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:27.870036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:27.870108 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:27.870207 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:27.872355 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.873028 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.875474 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.875933 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.876032 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.880415 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:27.880420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:27.880424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:27.880458 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:27.883227 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.884010 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.884024 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.884026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.884140 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.884561 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.884567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.884571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.884894 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:27.885052 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:27.885756 (+ 704us) server_negotiation.cc:299] Negotiation successful
0621 07:03:27.885979 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:03:27.887394 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.866291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44003 (local address 127.0.0.1:51076)
0621 07:03:27.867184 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.867230 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:27.869707 (+ 2477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:27.870478 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:27.870492 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:27.871351 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:27.872149 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.872176 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.873192 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.873200 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.875309 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.875327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.876168 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.876176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.876520 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.877289 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:27.877324 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:27.880145 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:27.883388 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.883397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.883415 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.883879 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.884250 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.884256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.884260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.884441 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.885022 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:27.885030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:27.885386 (+ 356us) client_negotiation.cc:769] Sending connection context
0621 07:03:27.885646 (+ 260us) client_negotiation.cc:240] Negotiation successful
0621 07:03:27.886183 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":417,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.252@KRBTEST.COM" created.
I20260621 07:03:27.889364 25558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:27 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:27.906040 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.890315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.2.245.253:39723)
0621 07:03:27.890866 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:27.890872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:27.890897 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:03:27.890983 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:27.890996 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:27.891208 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:27.891323 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:27.892322 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.893537 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.895386 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.895941 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.896035 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.900112 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:27.900123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:27.900135 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:27.900167 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:27.903121 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.903956 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.903962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.903964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.904084 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:27.904505 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:27.904508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:27.904510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:27.904853 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:27.905013 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:27.905424 (+ 411us) server_negotiation.cc:299] Negotiation successful
0621 07:03:27.905749 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":397,"thread_start_us":270,"threads_started":1}
I20260621 07:03:27.906121 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.890045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44900)
0621 07:03:27.890195 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.890209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:27.890288 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:27.891365 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:27.891376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:27.891894 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:27.892140 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.892154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.893685 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.893694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.895240 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:27.895266 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:27.896119 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:27.896128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:27.896350 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:27.896981 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:27.897010 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:27.899902 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:27.903249 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.903256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.903263 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.903837 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.904188 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:27.904195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:27.904202 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:27.904400 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:27.904985 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:27.904992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:27.905149 (+ 157us) client_negotiation.cc:769] Sending connection context
0621 07:03:27.905308 (+ 159us) client_negotiation.cc:240] Negotiation successful
0621 07:03:27.905627 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:03:27.908569 25558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:27.910034 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.909416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33150)
0621 07:03:27.909630 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.909795 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260621 07:03:27.910369 25510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:27.912128 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.911651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33152)
0621 07:03:27.911843 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.911963 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260621 07:03:27.912362 25558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
I20260621 07:03:27.923660 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:43681
--webserver_interface=127.2.245.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:03:27.961710 25558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 1
I20260621 07:03:27.963056 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.962453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33154)
0621 07:03:27.962687 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.962826 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260621 07:03:27.964506 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:27.964058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33168)
0621 07:03:27.964270 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:27.964369 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260621 07:03:27.964838 25558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.033098 25558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 2
I20260621 07:03:28.034439 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.033818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33184)
0621 07:03:28.034081 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.034200 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260621 07:03:28.035125 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } attempt: 5
I20260621 07:03:28.035825 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.035336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33186)
0621 07:03:28.035537 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.035633 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260621 07:03:28.036114 25558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:28.048672 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.035784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.2.245.253:39723)
0621 07:03:28.035934 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.035940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.036292 (+ 352us) server_negotiation.cc:407] Connection header received
0621 07:03:28.036361 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.036365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.036435 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.036542 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.037348 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.038119 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.039391 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.039828 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.039961 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.043462 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.043466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.043470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.043500 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.046088 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.046737 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.046743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.046747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.046866 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.047267 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.047272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.047276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.047555 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.047736 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.048122 (+ 386us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.048341 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:03:28.048677 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.035777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44908)
0621 07:03:28.036021 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.036040 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.036176 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.036579 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.036587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.037007 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.037187 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.037197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.038267 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.038273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.039225 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.039244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.039978 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.039983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.040200 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.040841 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.040858 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.043266 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.046231 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.046237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.046241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.046618 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.046997 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.047004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.047010 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.047151 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.047702 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.047707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.047874 (+ 167us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.048012 (+ 138us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.048297 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260621 07:03:28.049638 25484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:28.050653 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.050290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33194)
0621 07:03:28.050452 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.050532 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:03:28.051618 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.051302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33202)
0621 07:03:28.051445 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.051521 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260621 07:03:28.051832 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.103065 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 1
I20260621 07:03:28.104224 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.103732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33214)
0621 07:03:28.103954 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.104056 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260621 07:03:28.105316 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.104998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33226)
0621 07:03:28.105149 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.105212 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260621 07:03:28.105526 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.127369 25558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 3
I20260621 07:03:28.128626 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.128142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33230)
0621 07:03:28.128363 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.128457 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260621 07:03:28.129843 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.129416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33242)
0621 07:03:28.129637 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.129722 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260621 07:03:28.130107 25558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.173772 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 2
I20260621 07:03:28.174986 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.174438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33254)
0621 07:03:28.174666 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.174764 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:03:28.176090 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.175670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33262)
0621 07:03:28.175871 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.175960 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260621 07:03:28.176342 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
W20260621 07:03:28.265182 25565 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:28.265532 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:28.265655 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:28.265787 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:28.273367 25565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:28.273478 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260621 07:03:28.273350 25558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 4
W20260621 07:03:28.273561 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:28.273631 25565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:28.273717 25565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:28.274572 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.274092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33270)
0621 07:03:28.274327 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.274412 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260621 07:03:28.275624 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.275287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33282)
0621 07:03:28.275434 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.275516 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 07:03:28.275888 25558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.282577 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 3
I20260621 07:03:28.283761 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.283203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33290)
0621 07:03:28.283471 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.283577 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260621 07:03:28.284715 25565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:44003,127.2.245.253:39723,127.2.245.252:43681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:43681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.245.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:28.284902 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.284452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33300)
0621 07:03:28.284667 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.284769 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260621 07:03:28.285095 25484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:43681: Network error: Client connection negotiation failed: client connection to 127.2.245.252:43681: connect: Connection refused (error 111)
I20260621 07:03:28.286178 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:28.287496 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:28.299573 25571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:28.299805 25573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:28.300355 25570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:28.304759 25565 server_base.cc:1034] running on GCE node
I20260621 07:03:28.305601 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:28.307117 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:28.308363 25565 hybrid_clock.cc:648] HybridClock initialized: now 1782025408308297 us; error 46 us; skew 500 ppm
I20260621 07:03:28.308699 25565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:28.311946 25565 init.cc:377] Logged in from keytab as kudu/127.2.245.252@KRBTEST.COM (short username kudu)
I20260621 07:03:28.313591 25565 webserver.cc:460] Webserver started at http://127.2.245.252:37751/ using document root <none> and password file <none>
I20260621 07:03:28.314296 25565 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:28.314386 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:28.314646 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:28.317744 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data/instance:
uuid: "dadef48fbae941188081640f6b7530a6"
format_stamp: "Formatted at 2026-06-21 07:03:28 on dist-test-slave-9zdv"
server_key: "bb5dfd128e42560dcb22e66adc2bfcbc"
server_key_iv: "87696aa66fa14cbe056dbd25cd91c649"
server_key_version: "encryptionkey@0"
I20260621 07:03:28.318567 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal/instance:
uuid: "dadef48fbae941188081640f6b7530a6"
format_stamp: "Formatted at 2026-06-21 07:03:28 on dist-test-slave-9zdv"
server_key: "bb5dfd128e42560dcb22e66adc2bfcbc"
server_key_iv: "87696aa66fa14cbe056dbd25cd91c649"
server_key_version: "encryptionkey@0"
I20260621 07:03:28.323763 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260621 07:03:28.327136 25580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:28.328550 25565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:03:28.328727 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "dadef48fbae941188081640f6b7530a6"
format_stamp: "Formatted at 2026-06-21 07:03:28 on dist-test-slave-9zdv"
server_key: "bb5dfd128e42560dcb22e66adc2bfcbc"
server_key_iv: "87696aa66fa14cbe056dbd25cd91c649"
server_key_version: "encryptionkey@0"
I20260621 07:03:28.328979 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:28.346010 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:28.350433 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:28.350728 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:28.370103 25565 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.252:43681
I20260621 07:03:28.370110 25631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.252:43681 every 8 connection(s)
I20260621 07:03:28.371559 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
I20260621 07:03:28.375233 25632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:28.379598 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25565
I20260621 07:03:28.379783 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782025057166075-3031-0/minicluster-data/master-2/wal/instance
I20260621 07:03:28.380193 3031 external_mini_cluster.cc:1428] Setting key 9177d738a4687c27e108cc40f601d696
I20260621 07:03:28.381922 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:03:28.399922 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.384620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.2.245.254:44003)
0621 07:03:28.384793 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.384797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.385541 (+ 744us) server_negotiation.cc:407] Connection header received
0621 07:03:28.385969 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.385976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.386051 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.386173 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.387549 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.388238 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.389784 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.390214 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.390344 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.393739 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.393745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.393748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.393780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.396785 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.397585 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.397593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.397598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.397739 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.398147 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.398154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.398157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.398423 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.398583 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.399337 (+ 754us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.399555 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:03:28.401305 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.382010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.2.245.252:43681)
0621 07:03:28.382890 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.382903 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.382986 (+ 83us) server_negotiation.cc:407] Connection header received
0621 07:03:28.385383 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.385408 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.385768 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.385993 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.387573 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.388785 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.390024 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.390545 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.390681 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.394751 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.394759 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.394770 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.394799 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.397756 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.398469 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.398484 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.398487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.398617 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.399066 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.399073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.399076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.399419 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.399736 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.400180 (+ 444us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.400445 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"spinlock_wait_cycles":28416,"thread_start_us":368,"threads_started":1}
I20260621 07:03:28.401298 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.384643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:44003 (local address 127.0.0.1:51082)
0621 07:03:28.385395 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.385436 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.385726 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.386375 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.386380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.387034 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.387393 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.387409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.388405 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.388410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.389634 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.389646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.390463 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.390475 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.390651 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.391375 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.391397 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.393534 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.396926 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.396938 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.396954 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.397446 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.397855 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.397861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.397865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.398039 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.398561 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.398569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.398926 (+ 357us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.399249 (+ 323us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.399777 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":348,"threads_started":1}
I20260621 07:03:28.403133 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:28.416193 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.404019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.2.245.253:39723)
0621 07:03:28.404198 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.404204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.404223 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:03:28.404282 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.404287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.404365 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.404466 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.405139 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.405815 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.406997 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.407423 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.407560 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.410594 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.410599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.410602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.410635 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.413474 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.414273 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.414276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.414279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.414419 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.414897 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.414900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.414902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.415221 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.415340 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.415757 (+ 417us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.415919 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 07:03:28.416338 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.403797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:39723 (local address 127.0.0.1:44918)
0621 07:03:28.403981 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.404000 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.404100 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.404489 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.404494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.404852 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.404999 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.405007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.405949 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.405957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.406833 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.406843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.407567 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.407573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.407779 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.408362 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.408378 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.410404 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.413606 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.413612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.413616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.414151 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.414533 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.414541 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.414544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.414765 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.415339 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.415344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.415509 (+ 165us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.415665 (+ 156us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.415968 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:03:28.417311 25632 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:28.419209 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.404393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.2.245.254:44003)
0621 07:03:28.404550 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.404555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.404571 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:03:28.404664 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.404669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.404734 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.404829 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.405981 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.406707 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.408013 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.408443 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.408577 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.413092 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.413097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.413101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.413132 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.416243 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.416979 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.416984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.416987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.417135 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.417541 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.417547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.417551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.417917 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.418196 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.418472 (+ 276us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.418699 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:28.430779 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.418171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33332)
0621 07:03:28.418390 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.418410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.418498 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.418881 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.418885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.419365 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.419536 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.419545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.420660 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.420665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.421587 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.421598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.422589 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.422593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.422810 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.423442 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.423459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.425388 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.428356 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.428363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.428366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.428728 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.429133 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.429141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.429145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.429275 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.429751 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.429756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.429895 (+ 139us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.430055 (+ 160us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.430402 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":3}
I20260621 07:03:28.431331 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.418438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.2.245.252:43681)
0621 07:03:28.418582 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.418587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.418605 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:03:28.418655 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.418660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.418721 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.418842 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.419674 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.420534 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.421724 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.422424 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.422563 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.425576 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.425581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.425584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.425610 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.428198 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.428842 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.428850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.428853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.429019 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.429372 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.429380 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.429383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.429641 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.429795 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.430132 (+ 337us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.430651 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":55}
I20260621 07:03:28.435679 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.420867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.2.245.253:39723)
0621 07:03:28.421033 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.421038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.421055 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:03:28.421167 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.421172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.421239 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.421342 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.422870 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.423811 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.424984 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.425395 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.425530 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.429814 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.429819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.429822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.429854 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.432962 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.433633 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.433643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.433645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.433833 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.434236 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.434241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.434244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.434512 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.434809 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.435125 (+ 316us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.435365 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:03:28.436662 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: Bootstrap starting.
I20260621 07:03:28.440660 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:28.440740 25632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:28.441808 25632 log.cc:824] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:28.444670 25632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: No bootstrap required, opened a new log
I20260621 07:03:28.443392 25459 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f0cc3f9c4d774925960cf58706571d0a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51084
I20260621 07:03:28.451663 25632 raft_consensus.cc:348] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.452008 25632 raft_consensus.cc:729] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dadef48fbae941188081640f6b7530a6, State: Initialized, Role: FOLLOWER
I20260621 07:03:28.452437 25632 consensus_queue.cc:260] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.453269 25638 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.453574 25638 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.454502 25632 sys_catalog.cc:564] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:28.463044 25484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 4
W20260621 07:03:28.463937 25649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:03:28.464078 25649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:03:28.462128 25565 master_runner.cc:186] Error getting master registration for 127.2.245.254:44003: OK, instance_id { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" instance_seqno: 1782025407324234 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:03:28.474821 25565 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:03:28.479581 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.463690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33342)
0621 07:03:28.463951 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.463975 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.464094 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.464537 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.464541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.465134 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.465344 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.465356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.466434 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.466441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.467975 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.467990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.468836 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.468843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.469049 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.469721 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.469746 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.472488 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.476542 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.476549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.476555 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.477113 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.477534 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.477539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.477542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.477750 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.478493 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.478498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.478698 (+ 200us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.478852 (+ 154us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.479172 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260621 07:03:28.479808 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.463893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.2.245.252:43681)
0621 07:03:28.464071 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.464077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.464097 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:03:28.464308 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.464314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.464388 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.464517 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.465501 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.466278 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.468114 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.468655 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.468815 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.472669 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.472675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.472692 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.472722 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.476364 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.477242 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.477250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.477254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.477411 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.477864 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.477883 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.477889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.478354 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.478514 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.479167 (+ 653us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.479446 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:03:28.486133 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: Bootstrap starting.
I20260621 07:03:28.490420 25484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:28.491886 25484 log.cc:824] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:28.495224 25484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: No bootstrap required, opened a new log
I20260621 07:03:28.500303 25484 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.500651 25484 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0cc3f9c4d774925960cf58706571d0a, State: Initialized, Role: FOLLOWER
I20260621 07:03:28.501106 25484 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.502065 25652 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.502431 25652 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.503343 25484 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:28.503543 25459 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260621 07:03:28.503794 25459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:28.507543 25459 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.510789 25607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0cc3f9c4d774925960cf58706571d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dadef48fbae941188081640f6b7530a6"
I20260621 07:03:28.511399 25607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:28.518342 25607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0cc3f9c4d774925960cf58706571d0a in term 1.
I20260621 07:03:28.519563 25436 leader_election.cc:302] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dadef48fbae941188081640f6b7530a6, f0cc3f9c4d774925960cf58706571d0a; no voters:
I20260621 07:03:28.520220 25652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:28.513461 25417 master_runner.cc:186] Error getting master registration for 127.2.245.253:39723: OK, instance_id { permanent_uuid: "265796c64ef8441cba127e7470656cfc" instance_seqno: 1782025407843355 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:03:28.525399 25417 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:03:28.525161 25459 leader_election.cc:288] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [CANDIDATE]: Term 1 election: Requested vote from peers 265796c64ef8441cba127e7470656cfc (127.2.245.253:39723), dadef48fbae941188081640f6b7530a6 (127.2.245.252:43681)
I20260621 07:03:28.526355 25652 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [term 1 LEADER]: Becoming Leader. State: Replica: f0cc3f9c4d774925960cf58706571d0a, State: Running, Role: LEADER
I20260621 07:03:28.527038 25652 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.531275 25665 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0cc3f9c4d774925960cf58706571d0a. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.531764 25665 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: This master's current role is: LEADER
W20260621 07:03:28.536589 25663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:03:28.536720 25663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:03:28.543347 25669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:28.546068 25669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:28.562215 25607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [term 1 FOLLOWER]: Refusing update from remote peer f0cc3f9c4d774925960cf58706571d0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260621 07:03:28.563366 25438 tablet.cc:2369] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:03:28.563524 25665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:03:28.575095 25558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } attempt: 5
I20260621 07:03:28.593466 25638 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0cc3f9c4d774925960cf58706571d0a. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.593852 25638 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.600764 25672 mvcc.cc:205] Tried to move back new op lower bound from 7299176073456963584 to 7299176073332752384. Current Snapshot: MvccSnapshot[applied={T|T < 7299176073456963584}]
I20260621 07:03:28.617199 25638 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.617511 25638 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.620249 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.557588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.2.245.254:44003)
0621 07:03:28.557785 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.557792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.571349 (+ 13557us) server_negotiation.cc:407] Connection header received
0621 07:03:28.574059 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.574065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.574154 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.574237 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.578484 (+ 4247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.579617 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.583811 (+ 4194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.584564 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.584689 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.611225 (+ 26536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.611234 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.611241 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.611301 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.615856 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.616924 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.616931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.616935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.617190 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.617825 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.617834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.617839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.618293 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.618608 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.619375 (+ 767us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.619695 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Jun 21 07:03:28 dist-test-slave-9zdv krb5kdc[25404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:03:28.625294 25665 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.625586 25665 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:28.628434 25669 catalog_manager.cc:1347] Generated new cluster ID: 85fcf0a7d8dd4d54ae0091b7a03e162c
I20260621 07:03:28.628539 25669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:28.634913 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.565826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.2.245.252:43681)
0621 07:03:28.565992 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.565998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.574863 (+ 8865us) server_negotiation.cc:407] Connection header received
0621 07:03:28.575739 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.575746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.575875 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.575958 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.594091 (+ 18133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.595421 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.597394 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.598126 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.598279 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.625500 (+ 27221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.625508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.625514 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.625560 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.629396 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.632336 (+ 2940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.632344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.632349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.632500 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.632963 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.632968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.632972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.633352 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.633626 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.634203 (+ 577us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.634445 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":52}
I20260621 07:03:28.634997 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.575707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:43681 (local address 127.0.0.1:33360)
0621 07:03:28.576069 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:28.576088 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:28.576200 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:28.606328 (+ 30128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:28.606334 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:28.606861 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:28.607096 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.607109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.611243 (+ 4134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.611250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.612755 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:28.612773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.618526 (+ 5753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.618533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:28.618805 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.619457 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:28.619481 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:28.622611 (+ 3130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:28.631072 (+ 8461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.631078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.631083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.631762 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.633049 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:28.633054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:28.633058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:28.633230 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:28.633927 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:28.633933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:28.634106 (+ 173us) client_negotiation.cc:769] Sending connection context
0621 07:03:28.634251 (+ 145us) client_negotiation.cc:240] Negotiation successful
0621 07:03:28.634551 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236}
I20260621 07:03:28.636592 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.576437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.2.245.252:43681)
0621 07:03:28.596417 (+ 19980us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.596423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.596448 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:03:28.596505 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.596510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.596587 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.607294 (+ 10707us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.607598 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.610825 (+ 3227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.613111 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.613805 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.622809 (+ 9004us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.623097 (+ 288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.623103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.623108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.623141 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.630890 (+ 7749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.632698 (+ 1808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.632703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.632707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.632868 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.633386 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.633391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.633395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.633791 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.635773 (+ 1982us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.635900 (+ 127us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.636104 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19826,"thread_start_us":20233,"threads_started":1}
I20260621 07:03:28.642572 25558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: Bootstrap starting.
I20260621 07:03:28.646811 25558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:28.648427 25558 log.cc:824] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:28.649796 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:28.571988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.2.245.253:39723)
0621 07:03:28.572163 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.572169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.595415 (+ 23246us) server_negotiation.cc:407] Connection header received
0621 07:03:28.599849 (+ 4434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.599857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.599970 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.600075 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.604545 (+ 4470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.605915 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.621271 (+ 15356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:28.621904 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:28.622034 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:28.628664 (+ 6630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:28.628670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:28.628674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:28.628713 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:28.632517 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.647096 (+ 14579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.647106 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.647112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.647330 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:28.647887 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:28.647893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:28.647896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:28.648312 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:28.648569 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:28.649171 (+ 602us) server_negotiation.cc:299] Negotiation successful
0621 07:03:28.649439 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 07:03:28.671769 25558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: No bootstrap required, opened a new log
I20260621 07:03:28.683960 25669 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:28.691777 25669 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:28.694855 25558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.695708 25558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265796c64ef8441cba127e7470656cfc, State: Initialized, Role: FOLLOWER
I20260621 07:03:28.696610 25558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } }
I20260621 07:03:28.707942 25533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0cc3f9c4d774925960cf58706571d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "265796c64ef8441cba127e7470656cfc"
I20260621 07:03:28.708456 25533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:28.713620 25533 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0cc3f9c4d774925960cf58706571d0a in term 1.
I20260621 07:03:28.714435 25532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [term 1 FOLLOWER]: Refusing update from remote peer f0cc3f9c4d774925960cf58706571d0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:03:28.716236 25665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:03:28.717474 25669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a: Generated new TSK 0
I20260621 07:03:28.718590 25669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:28.721676 25558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:28.773792 25682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.773870 25683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0cc3f9c4d774925960cf58706571d0a. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.774380 25683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.774377 25682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.774849 25691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:28.775455 25691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:28.810643 25702 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260621 07:03:28.835470 25487 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.828240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.2.245.254:44003)
0621 07:03:28.828467 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.828473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.832188 (+ 3715us) server_negotiation.cc:407] Connection header received
0621 07:03:28.833753 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.833760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.833851 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.833926 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.835197 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51106: BlockingRecv error: recv got EOF from 127.0.0.1:51106 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 07:03:28.837713 25677 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.831455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.2.245.252:43681)
0621 07:03:28.831649 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.831656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.836054 (+ 4398us) server_negotiation.cc:407] Connection header received
0621 07:03:28.836310 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.836316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.836413 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.836485 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.837454 (+ 969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33370: BlockingRecv error: recv got EOF from 127.0.0.1:33370 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260621 07:03:28.840674 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.838918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.2.245.253:39723)
0621 07:03:28.839144 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.839150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.839193 (+ 43us) server_negotiation.cc:407] Connection header received
0621 07:03:28.839374 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.839380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.839466 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.839570 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.840456 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44934: BlockingRecv error: recv got EOF from 127.0.0.1:44934 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260621 07:03:28.848193 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.845431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.2.245.253:39723)
0621 07:03:28.845591 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.845600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.846667 (+ 1067us) server_negotiation.cc:407] Connection header received
0621 07:03:28.846879 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.846885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.847033 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.847138 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.847902 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44936: BlockingRecv error: recv got EOF from 127.0.0.1:44936 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260621 07:03:28.885562 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.883333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.2.245.253:39723)
0621 07:03:28.883521 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.883529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.884044 (+ 515us) server_negotiation.cc:407] Connection header received
0621 07:03:28.884250 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.884256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.884359 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.884472 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.885342 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44950: BlockingRecv error: recv got EOF from 127.0.0.1:44950 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260621 07:03:28.895220 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.892884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.2.245.253:39723)
0621 07:03:28.893064 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.893069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.893638 (+ 569us) server_negotiation.cc:407] Connection header received
0621 07:03:28.893870 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.893876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.893983 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.894087 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.894932 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44966: BlockingRecv error: recv got EOF from 127.0.0.1:44966 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260621 07:03:28.906302 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.903911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44978 (local address 127.2.245.253:39723)
0621 07:03:28.904129 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.904135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.904639 (+ 504us) server_negotiation.cc:407] Connection header received
0621 07:03:28.904940 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.904946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.905056 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.905169 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.906046 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44978: BlockingRecv error: recv got EOF from 127.0.0.1:44978 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260621 07:03:28.920415 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.918027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.2.245.253:39723)
0621 07:03:28.918220 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.918227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.918809 (+ 582us) server_negotiation.cc:407] Connection header received
0621 07:03:28.919051 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.919057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.919149 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.919258 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.920154 (+ 896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44988: BlockingRecv error: recv got EOF from 127.0.0.1:44988 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260621 07:03:28.942655 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.940244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.2.245.253:39723)
0621 07:03:28.940452 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.940460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.941008 (+ 548us) server_negotiation.cc:407] Connection header received
0621 07:03:28.941232 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.941240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.941349 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.941457 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.942409 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45002: BlockingRecv error: recv got EOF from 127.0.0.1:45002 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260621 07:03:28.983629 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:28.981151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.2.245.253:39723)
0621 07:03:28.981369 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:28.981375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:28.981933 (+ 558us) server_negotiation.cc:407] Connection header received
0621 07:03:28.982169 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:28.982177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:28.982278 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:28.982397 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:28.983366 (+ 969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45016: BlockingRecv error: recv got EOF from 127.0.0.1:45016 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 07:03:29.053412 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:29.051182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.2.245.253:39723)
0621 07:03:29.051464 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.051472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:29.051904 (+ 432us) server_negotiation.cc:407] Connection header received
0621 07:03:29.052126 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:29.052130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:29.052201 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:29.052288 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:29.053193 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45018: BlockingRecv error: recv got EOF from 127.0.0.1:45018 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260621 07:03:29.116698 25665 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0cc3f9c4d774925960cf58706571d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0cc3f9c4d774925960cf58706571d0a" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 44003 } } peers { permanent_uuid: "265796c64ef8441cba127e7470656cfc" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 39723 } } peers { permanent_uuid: "dadef48fbae941188081640f6b7530a6" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 43681 } } }
I20260621 07:03:29.117079 25665 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cc3f9c4d774925960cf58706571d0a [sys.catalog]: This master's current role is: LEADER
W20260621 07:03:29.187201 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:29.185179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.2.245.253:39723)
0621 07:03:29.185440 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.185448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:29.185773 (+ 325us) server_negotiation.cc:407] Connection header received
0621 07:03:29.186016 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:29.186021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:29.186094 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:29.186210 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:29.186981 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45030: BlockingRecv error: recv got EOF from 127.0.0.1:45030 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260621 07:03:29.450680 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:29.448854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.2.245.253:39723)
0621 07:03:29.449103 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.449109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:29.449283 (+ 174us) server_negotiation.cc:407] Connection header received
0621 07:03:29.449548 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:29.449558 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:29.449633 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:29.449758 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:29.450505 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45036: BlockingRecv error: recv got EOF from 127.0.0.1:45036 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260621 07:03:29.466009 25649 catalog_manager.cc:1259] Loaded cluster ID: 85fcf0a7d8dd4d54ae0091b7a03e162c
I20260621 07:03:29.466107 25649 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: loading cluster ID for follower catalog manager: success
I20260621 07:03:29.468688 25649 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: acquiring CA information for follower catalog manager: success
I20260621 07:03:29.470435 25649 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dadef48fbae941188081640f6b7530a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:03:29.815160 25702 catalog_manager.cc:1259] Loaded cluster ID: 85fcf0a7d8dd4d54ae0091b7a03e162c
I20260621 07:03:29.815389 25702 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: loading cluster ID for follower catalog manager: success
I20260621 07:03:29.818166 25702 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: acquiring CA information for follower catalog manager: success
I20260621 07:03:29.820056 25702 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 265796c64ef8441cba127e7470656cfc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:03:29.977178 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:29.968497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.2.245.253:39723)
0621 07:03:29.969301 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.969311 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:29.969854 (+ 543us) server_negotiation.cc:407] Connection header received
0621 07:03:29.970038 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:29.970044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:29.970259 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:29.970424 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:29.971179 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:29.972527 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:29.974567 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:29.975134 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:29.975244 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:29.975845 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:29.976461 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:29.976578 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:29.976724 (+ 146us) server_negotiation.cc:299] Negotiation successful
0621 07:03:29.976802 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":411,"threads_started":1}
I20260621 07:03:29.990598 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:29.984487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.2.245.252:43681)
0621 07:03:29.985144 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.985153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:29.985173 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:03:29.985301 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:29.985304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:29.985460 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:29.985640 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:29.986161 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:29.986899 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:29.988311 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:29.988736 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:29.988878 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:29.989452 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:29.989980 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:29.990156 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:29.990209 (+ 53us) server_negotiation.cc:299] Negotiation successful
0621 07:03:29.990290 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":418,"threads_started":1}
I20260621 07:03:30.020893 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:29.998962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.2.245.252:43681)
0621 07:03:29.999199 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:29.999206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:30.011751 (+ 12545us) server_negotiation.cc:407] Connection header received
0621 07:03:30.012153 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:30.012160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:30.012295 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:30.012404 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:30.013400 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.014882 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.018027 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.018770 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.018877 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:30.019859 (+ 982us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:30.020030 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:30.020105 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:30.020485 (+ 380us) server_negotiation.cc:299] Negotiation successful
0621 07:03:30.020588 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:03:30.023727 25776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:29.998828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.2.245.254:44003)
0621 07:03:30.014548 (+ 15720us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:30.014555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:30.014577 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:03:30.014693 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:30.014699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:30.014914 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:30.015961 (+ 1047us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:30.016138 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.017264 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.019764 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.020439 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.021293 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:30.021505 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:30.022895 (+ 1390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:30.023091 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:30.023218 (+ 127us) server_negotiation.cc:299] Negotiation successful
0621 07:03:30.023374 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15588,"thread_start_us":16273,"threads_started":1}
I20260621 07:03:30.026362 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:30.011964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.2.245.253:39723)
0621 07:03:30.012152 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:30.012158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:30.018885 (+ 6727us) server_negotiation.cc:407] Connection header received
0621 07:03:30.019423 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:30.019430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:30.019566 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:30.019641 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:30.020644 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.021784 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.024097 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:30.024788 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:30.024935 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:30.025541 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:30.025698 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:30.025816 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:30.025986 (+ 170us) server_negotiation.cc:299] Negotiation successful
0621 07:03:30.026070 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:03:30.030000 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25417
I20260621 07:03:30.031091 25479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:30.159271 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25417
W20260621 07:03:30.180698 25508 connection.cc:535] server connection from 127.0.0.1:44908 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260621 07:03:30.181152 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25491
I20260621 07:03:30.182420 25553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:30.297822 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25491
I20260621 07:03:30.318625 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25565
I20260621 07:03:30.319823 25627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:30.427249 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25565
2026-06-21T07:03:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4900 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:03:30 dist-test-slave-9zdv krb5kdc[25785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:03:30 dist-test-slave-9zdv krb5kdc[25785](info): set up 2 sockets
Jun 21 07:03:30 dist-test-slave-9zdv krb5kdc[25785](info): commencing operation
krb5kdc: starting...
W20260621 07:03:31.556041 3031 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/test-user.keytab.
Jun 21 07:03:31 dist-test-slave-9zdv krb5kdc[25785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.245.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
Entry for principal kudu/127.2.245.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab.
I20260621 07:03:31.828480 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43665
--webserver_interface=127.2.245.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:03:32.163425 25798 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:32.163805 25798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:32.163939 25798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:32.164078 25798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:32.172082 25798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:32.172211 25798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:32.172331 25798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:32.172420 25798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:32.172528 25798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:32.182498 25798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.254.keytab
--principal=kudu/127.2.245.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.254:43665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.245.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:32.184266 25798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:32.185627 25798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:32.197793 25806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:32.197822 25804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:32.197849 25803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:32.198662 25798 server_base.cc:1034] running on GCE node
I20260621 07:03:32.199517 25798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:32.200898 25798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:32.202160 25798 hybrid_clock.cc:648] HybridClock initialized: now 1782025412202080 us; error 40 us; skew 500 ppm
I20260621 07:03:32.202555 25798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:32 dist-test-slave-9zdv krb5kdc[25785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:32.206068 25798 init.cc:377] Logged in from keytab as kudu/127.2.245.254@KRBTEST.COM (short username kudu)
I20260621 07:03:32.207742 25798 webserver.cc:460] Webserver started at http://127.2.245.254:45855/ using document root <none> and password file <none>
I20260621 07:03:32.208496 25798 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:32.208621 25798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:32.208972 25798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:32.212160 25798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data/instance:
uuid: "19c513a0a8ec4d31b1b6a36830b0618b"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "5d491b7147230818a949d80943c9332e"
server_key_iv: "ba44b37535437936e975806059160151"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.212983 25798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance:
uuid: "19c513a0a8ec4d31b1b6a36830b0618b"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "5d491b7147230818a949d80943c9332e"
server_key_iv: "ba44b37535437936e975806059160151"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.218483 25798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260621 07:03:32.221995 25813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:32.223431 25798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:03:32.223627 25798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal
uuid: "19c513a0a8ec4d31b1b6a36830b0618b"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "5d491b7147230818a949d80943c9332e"
server_key_iv: "ba44b37535437936e975806059160151"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.223891 25798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:32.268301 25798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:32.272903 25798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:32.273207 25798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:32.292634 25798 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.254:43665
I20260621 07:03:32.292642 25864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.254:43665 every 8 connection(s)
I20260621 07:03:32.294029 25798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/data/info.pb
I20260621 07:03:32.295058 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25798
I20260621 07:03:32.295254 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-0/wal/instance
I20260621 07:03:32.295848 3031 external_mini_cluster.cc:1428] Setting key 7763315b6d0922328363f22369e31904
I20260621 07:03:32.299037 25865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:32.308583 25865 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.245.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.253@KRBTEST.COM" created.
Jun 21 07:03:32 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:32.334559 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.313475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.2.245.254:43665)
0621 07:03:32.314191 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:32.314199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:32.314262 (+ 63us) server_negotiation.cc:407] Connection header received
0621 07:03:32.315567 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:32.315573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:32.315822 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:32.316049 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:32.317362 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.318546 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.320116 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.320677 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.320881 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.326940 (+ 6059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:32.326971 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:32.326989 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:32.327040 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:32.330060 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.330797 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.330804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.330808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.331014 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.331411 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.331416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.331419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.331937 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:32.332154 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:32.332673 (+ 519us) server_negotiation.cc:299] Negotiation successful
0621 07:03:32.333051 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":519,"thread_start_us":299,"threads_started":1}
I20260621 07:03:32.334568 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.312129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43665 (local address 127.0.0.1:57906)
0621 07:03:32.312977 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.313023 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:32.315177 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:32.316059 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:32.316065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:32.316799 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:32.317141 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.317159 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.318711 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.318722 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.319976 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.319989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.320956 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.320962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.321171 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.321909 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:32.321930 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:32.326678 (+ 4748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:32.330224 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.330235 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.330247 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.330660 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.331130 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.331136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.331139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.331298 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.332081 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:32.332093 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:32.332299 (+ 206us) client_negotiation.cc:769] Sending connection context
0621 07:03:32.332540 (+ 241us) client_negotiation.cc:240] Negotiation successful
0621 07:03:32.333161 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":151,"thread_start_us":347,"threads_started":1}
I20260621 07:03:32.337854 25865 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:32.339006 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.338556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60854)
0621 07:03:32.338722 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.338833 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260621 07:03:32.339284 25816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
I20260621 07:03:32.340874 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.340519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60868)
0621 07:03:32.340703 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.340765 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260621 07:03:32.341071 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:41363: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
Entry for principal kudu/127.2.245.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab.
I20260621 07:03:32.358366 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:41363
--webserver_interface=127.2.245.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:03:32.390414 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } attempt: 1
I20260621 07:03:32.391721 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.391193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60882)
0621 07:03:32.391402 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.391565 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260621 07:03:32.392776 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.392422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60886)
0621 07:03:32.392567 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.392650 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260621 07:03:32.392973 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:41363: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
I20260621 07:03:32.461228 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } attempt: 2
I20260621 07:03:32.462455 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.461942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60902)
0621 07:03:32.462209 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.462302 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260621 07:03:32.463490 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.463127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60916)
0621 07:03:32.463269 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.463356 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 07:03:32.463657 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:41363: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
I20260621 07:03:32.555052 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } attempt: 3
I20260621 07:03:32.556373 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.555915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60930)
0621 07:03:32.556140 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.556238 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260621 07:03:32.557426 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.557063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60932)
0621 07:03:32.557207 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.557285 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260621 07:03:32.557600 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:41363: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
W20260621 07:03:32.699052 25872 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:32.699443 25872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:32.699568 25872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:32.699700 25872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260621 07:03:32.700839 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } attempt: 4
I20260621 07:03:32.702092 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.701487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60944)
0621 07:03:32.701817 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.701927 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260621 07:03:32.703224 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.702761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60958)
0621 07:03:32.702971 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.703069 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260621 07:03:32.703418 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.253:41363: Network error: Client connection negotiation failed: client connection to 127.2.245.253:41363: connect: Connection refused (error 111)
W20260621 07:03:32.707991 25872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:32.708081 25872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:03:32.708204 25872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:32.708292 25872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:32.708386 25872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:32.718665 25872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.253.keytab
--principal=kudu/127.2.245.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.253:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.245.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:32.720106 25872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:32.721429 25872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:32.733686 25878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:32.733718 25877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:32.733749 25880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:32.734560 25872 server_base.cc:1034] running on GCE node
I20260621 07:03:32.735394 25872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:32.736768 25872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:32.738039 25872 hybrid_clock.cc:648] HybridClock initialized: now 1782025412737971 us; error 42 us; skew 500 ppm
I20260621 07:03:32.738404 25872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:32 dist-test-slave-9zdv krb5kdc[25785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:32.741591 25872 init.cc:377] Logged in from keytab as kudu/127.2.245.253@KRBTEST.COM (short username kudu)
I20260621 07:03:32.743193 25872 webserver.cc:460] Webserver started at http://127.2.245.253:46663/ using document root <none> and password file <none>
I20260621 07:03:32.743914 25872 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:32.744052 25872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:32.744376 25872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:32.747489 25872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data/instance:
uuid: "4584e68482494705a2e692e612046cba"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "ac165c830f12ae51301532fa8b8494a6"
server_key_iv: "be225e6f3443e54058ace0b4915da070"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.748302 25872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal/instance:
uuid: "4584e68482494705a2e692e612046cba"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "ac165c830f12ae51301532fa8b8494a6"
server_key_iv: "be225e6f3443e54058ace0b4915da070"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.753557 25872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:03:32.757072 25887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:32.758384 25872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:03:32.758577 25872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal
uuid: "4584e68482494705a2e692e612046cba"
format_stamp: "Formatted at 2026-06-21 07:03:32 on dist-test-slave-9zdv"
server_key: "ac165c830f12ae51301532fa8b8494a6"
server_key_iv: "be225e6f3443e54058ace0b4915da070"
server_key_version: "encryptionkey@0"
I20260621 07:03:32.758807 25872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:32.777014 25872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:32.781699 25872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:32.781993 25872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:32.801460 25872 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.253:41363
I20260621 07:03:32.801471 25938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.253:41363 every 8 connection(s)
I20260621 07:03:32.802894 25872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/data/info.pb
I20260621 07:03:32.805255 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25872
I20260621 07:03:32.806717 25939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:32.810062 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-1/wal/instance
I20260621 07:03:32.810546 3031 external_mini_cluster.cc:1428] Setting key 863c76a92538847b1a3f18d0a1aebe8c
I20260621 07:03:32.813306 25939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:32 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.245.252@KRBTEST.COM; defaulting to no policy
I20260621 07:03:32.834502 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.816112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.2.245.254:43665)
0621 07:03:32.816289 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:32.816294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:32.817607 (+ 1313us) server_negotiation.cc:407] Connection header received
0621 07:03:32.819665 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:32.819672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:32.819768 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:32.819880 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:32.821407 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.822082 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.823725 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.824161 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.824293 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.828271 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:32.828276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:32.828280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:32.828313 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:32.831354 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.832124 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.832129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.832132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.832244 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.832688 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.832694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.832699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.832994 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:32.833178 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:32.833927 (+ 749us) server_negotiation.cc:299] Negotiation successful
0621 07:03:32.834162 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:03:32.835640 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.816648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43665 (local address 127.0.0.1:57920)
0621 07:03:32.817434 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.817472 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:32.819378 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:32.820123 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:32.820132 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:32.820654 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:32.821203 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.821220 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.822245 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.822249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.823564 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.823576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.824394 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.824402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.824643 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.825337 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:32.825362 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:32.828019 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:32.831508 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.831517 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.831536 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.831984 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.832360 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.832365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.832368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.832565 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.833130 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:32.833141 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:32.833517 (+ 376us) client_negotiation.cc:769] Sending connection context
0621 07:03:32.833833 (+ 316us) client_negotiation.cc:240] Negotiation successful
0621 07:03:32.834366 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.245.252@KRBTEST.COM" created.
I20260621 07:03:32.837704 25939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:32 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:32.853230 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.838699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.2.245.253:41363)
0621 07:03:32.839353 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:32.839359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:32.839384 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:03:32.839426 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:32.839432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:32.839631 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:32.839789 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:32.840575 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.841638 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.842836 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.843452 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.843602 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.847154 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:32.847166 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:32.847180 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:32.847215 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:32.850071 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.850932 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.851040 (+ 108us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.851043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.851158 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:32.851589 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:32.851592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:32.851594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:32.851942 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:32.852120 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:32.852563 (+ 443us) server_negotiation.cc:299] Negotiation successful
0621 07:03:32.852887 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":507,"thread_start_us":343,"threads_started":1}
I20260621 07:03:32.853317 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.838620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60968)
0621 07:03:32.838824 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.838841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:32.839024 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:32.839776 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:32.839781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:32.840262 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:32.840393 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.840401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.841776 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.841784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.842642 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:32.842653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:32.843622 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:32.843628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:32.843803 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:32.844436 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:32.844469 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:32.846906 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:32.850219 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.850227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.850234 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.850797 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.851276 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:32.851284 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:32.851290 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:32.851476 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:32.852097 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:32.852104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:32.852272 (+ 168us) client_negotiation.cc:769] Sending connection context
0621 07:03:32.852430 (+ 158us) client_negotiation.cc:240] Negotiation successful
0621 07:03:32.852766 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 07:03:32.855966 25939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:32.857291 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.856700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51740)
0621 07:03:32.856930 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.857078 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260621 07:03:32.857607 25889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:32.859323 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.858866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51750)
0621 07:03:32.859069 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.859172 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260621 07:03:32.859577 25939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
Entry for principal kudu/127.2.245.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab.
I20260621 07:03:32.870321 3031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
/tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:37101
--webserver_interface=127.2.245.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.245.212:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:03:32.908918 25939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 1
I20260621 07:03:32.910117 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.909664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51758)
0621 07:03:32.909860 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.909972 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260621 07:03:32.911154 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.910749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51774)
0621 07:03:32.910928 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.911062 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 07:03:32.911347 25939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:32.979615 25939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 2
I20260621 07:03:32.980772 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.980322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51790)
0621 07:03:32.980525 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.980622 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260621 07:03:32.981956 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:32.981540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51804)
0621 07:03:32.981706 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:32.981776 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260621 07:03:32.982511 25939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.002673 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } attempt: 5
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:33.017480 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.003497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60976)
0621 07:03:33.003687 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.003707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.003819 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.004217 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.004220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.004583 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.004773 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.004782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.006200 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.006203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.007131 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.007144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.008233 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.008237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.008394 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.009051 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.009068 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.011247 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.015140 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.015144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.015147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.015550 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.016020 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.016023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.016025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.016152 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.016749 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.016752 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.016875 (+ 123us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.016983 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.017207 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260621 07:03:33.017853 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.003588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.2.245.253:41363)
0621 07:03:33.003779 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.003784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.003809 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:03:33.003944 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.003950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.004055 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.004205 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.004951 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.006064 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.007287 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.008088 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.008223 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.011457 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.011463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.011469 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.011508 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.014994 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.015698 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.015705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.015710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.015907 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.016257 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.016262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.016265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.016637 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.016789 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.017141 (+ 352us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.017443 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:03:33.018846 25865 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:03:33.020028 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.019651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51818)
0621 07:03:33.019812 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.019897 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260621 07:03:33.021063 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.020623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51828)
0621 07:03:33.020846 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.020940 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260621 07:03:33.021293 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.072568 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 1
I20260621 07:03:33.073766 25939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 3
I20260621 07:03:33.074049 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.073357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51830)
0621 07:03:33.073633 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.073803 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260621 07:03:33.075030 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.074400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51832)
0621 07:03:33.074621 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.074750 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260621 07:03:33.075356 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.074864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51836)
0621 07:03:33.075096 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.075208 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260621 07:03:33.075590 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.076169 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.075769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51850)
0621 07:03:33.075994 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.076066 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260621 07:03:33.076388 25939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.143887 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 2
I20260621 07:03:33.145154 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.144555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51862)
0621 07:03:33.144816 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.144933 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260621 07:03:33.146314 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.145888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51864)
0621 07:03:33.146062 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.146162 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 07:03:33.146559 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
W20260621 07:03:33.211978 25946 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 07:03:33.212353 25946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:03:33.212473 25946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:03:33.212605 25946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:03:33.219638 25946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:03:33.219720 25946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260621 07:03:33.219601 25939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 4
W20260621 07:03:33.219820 25946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:03:33.219892 25946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:03:33.220014 25946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:03:33.220906 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.220331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51876)
0621 07:03:33.220625 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.220732 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260621 07:03:33.221968 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.221610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51892)
0621 07:03:33.221758 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.221852 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260621 07:03:33.222194 25939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.230587 25946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.245.212:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.245.254:43665,127.2.245.253:41363,127.2.245.252:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/krb5kdc/kudu_127.2.245.252.keytab
--principal=kudu/127.2.245.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.245.252:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.245.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 21 Jun 2026 05:57:27 UTC on ccycloud.quasar-zbapqx.root.comops.site
build id 4457935
ASAN enabled
I20260621 07:03:33.231933 25946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:03:33.233256 25946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:03:33.245126 25951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:33.245131 25954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:03:33.245167 25952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:03:33.246061 25946 server_base.cc:1034] running on GCE node
I20260621 07:03:33.246862 25946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:03:33.248270 25946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:03:33.249523 25946 hybrid_clock.cc:648] HybridClock initialized: now 1782025413249445 us; error 51 us; skew 500 ppm
I20260621 07:03:33.249872 25946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025413, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:03:33.252759 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 3
I20260621 07:03:33.253062 25946 init.cc:377] Logged in from keytab as kudu/127.2.245.252@KRBTEST.COM (short username kudu)
I20260621 07:03:33.253953 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.253414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51898)
0621 07:03:33.253625 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.253741 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260621 07:03:33.254904 25946 webserver.cc:460] Webserver started at http://127.2.245.252:36533/ using document root <none> and password file <none>
I20260621 07:03:33.255138 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.254577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51908)
0621 07:03:33.254858 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.254991 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260621 07:03:33.255326 25865 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.245.252:37101: Network error: Client connection negotiation failed: client connection to 127.2.245.252:37101: connect: Connection refused (error 111)
I20260621 07:03:33.255746 25946 fs_manager.cc:359] Metadata directory not provided
I20260621 07:03:33.255872 25946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:03:33.256219 25946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:03:33.259193 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data/instance:
uuid: "627114969a894eea9d38909cfe72f025"
format_stamp: "Formatted at 2026-06-21 07:03:33 on dist-test-slave-9zdv"
server_key: "c6c70913b185d74fd6e9f657c608294a"
server_key_iv: "ce19a01512c68ead9200423ebbd4821e"
server_key_version: "encryptionkey@0"
I20260621 07:03:33.259977 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal/instance:
uuid: "627114969a894eea9d38909cfe72f025"
format_stamp: "Formatted at 2026-06-21 07:03:33 on dist-test-slave-9zdv"
server_key: "c6c70913b185d74fd6e9f657c608294a"
server_key_iv: "ce19a01512c68ead9200423ebbd4821e"
server_key_version: "encryptionkey@0"
I20260621 07:03:33.265202 25946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260621 07:03:33.268594 25961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:03:33.269898 25946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260621 07:03:33.270076 25946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data,/tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal
uuid: "627114969a894eea9d38909cfe72f025"
format_stamp: "Formatted at 2026-06-21 07:03:33 on dist-test-slave-9zdv"
server_key: "c6c70913b185d74fd6e9f657c608294a"
server_key_iv: "ce19a01512c68ead9200423ebbd4821e"
server_key_version: "encryptionkey@0"
I20260621 07:03:33.270284 25946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:03:33.288338 25946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:03:33.292979 25946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:03:33.293289 25946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:03:33.313076 25946 rpc_server.cc:307] RPC server started. Bound to: 127.2.245.252:37101
I20260621 07:03:33.313095 26012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.245.252:37101 every 8 connection(s)
I20260621 07:03:33.314491 25946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/data/info.pb
I20260621 07:03:33.315953 3031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu as pid 25946
I20260621 07:03:33.316128 3031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV0VCJ_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782025057166075-3031-0/minicluster-data/master-2/wal/instance
I20260621 07:03:33.316572 3031 external_mini_cluster.cc:1428] Setting key eced23399baffd65fcc3dc7dec220360
I20260621 07:03:33.319026 26013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:03:33.327266 26013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025413, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
I20260621 07:03:33.339818 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.318338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.2.245.252:37101)
0621 07:03:33.319177 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.319192 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.319244 (+ 52us) server_negotiation.cc:407] Connection header received
0621 07:03:33.320798 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.320818 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.321182 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.321518 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.323343 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.324700 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.326500 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.327332 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.327565 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.332584 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.332591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.332601 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.332641 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.335631 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.336298 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.336306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.336315 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.336554 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.336992 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.336997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.337000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.337481 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.337775 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.338261 (+ 486us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.338659 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"spinlock_wait_cycles":22912,"thread_start_us":366,"threads_started":1}
I20260621 07:03:33.344658 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.330232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.254:43665 (local address 127.0.0.1:57930)
0621 07:03:33.330859 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.330910 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.331093 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.331505 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.331509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.332147 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.332386 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.332398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.333515 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.333520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.334756 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.334767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.335564 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.335572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.335752 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.336322 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.336339 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.338376 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.341820 (+ 3444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.341832 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.341839 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.342337 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.342785 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.342790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.342796 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.343043 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.343811 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.343840 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.344025 (+ 185us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.344170 (+ 145us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.344401 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":336,"threads_started":1}
I20260621 07:03:33.345185 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.330318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57930 (local address 127.2.245.254:43665)
0621 07:03:33.330542 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.330550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.331036 (+ 486us) server_negotiation.cc:407] Connection header received
0621 07:03:33.331218 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.331224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.331331 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.331453 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.332536 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.333337 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.334901 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.335385 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.335504 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.338646 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.338652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.338655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.338686 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.341674 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.342458 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.342466 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.342471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.342652 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.343176 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.343184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.343191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.343664 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.343843 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.344345 (+ 502us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.344659 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:03:33.348519 26013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.254@KRBTEST.COM
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025413, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
I20260621 07:03:33.361605 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.342968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.2.245.254:43665)
0621 07:03:33.345484 (+ 2516us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.345492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.345514 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:03:33.345573 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.345580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.345668 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.345788 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.346905 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.348087 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.349675 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.350304 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.350448 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.355199 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.355204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.355207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.355239 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.358498 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.359357 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.359363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.359367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.359533 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.359924 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.359942 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.359945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.360354 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.360635 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.360983 (+ 348us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.361208 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2368,"thread_start_us":4139,"threads_started":1}
I20260621 07:03:33.363396 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.349275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.253:41363 (local address 127.0.0.1:60984)
0621 07:03:33.349459 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.349475 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.349553 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.349884 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.349888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.350368 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.350567 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.350583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.351968 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.351972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.352818 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.352828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.353725 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.353732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.353947 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.354533 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.354550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.356777 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.360576 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.360583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.360587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.361091 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.361541 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.361548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.361554 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.361697 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.362288 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.362292 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.362553 (+ 261us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.362729 (+ 176us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.363058 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260621 07:03:33.364249 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.349340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.2.245.253:41363)
0621 07:03:33.349602 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.349611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.349631 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:03:33.349699 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.349703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.349761 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.349832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.350736 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.351802 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.352960 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.353541 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.353646 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.357005 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.357012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.357017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.357062 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.360451 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.361235 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.361241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.361243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.361415 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.361858 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.361865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.361872 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.362162 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.362325 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.362825 (+ 500us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.363075 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260621 07:03:33.365301 26013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.245.253@KRBTEST.COM
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025413, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.252@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:03:33.378655 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.363286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.2.245.253:41363)
0621 07:03:33.364190 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.364195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.364216 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:03:33.364258 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.364261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.364309 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.364438 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.366066 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.367027 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.368271 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.368721 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.368857 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.373117 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.373122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.373125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.373154 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.376098 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.376747 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.376754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.376757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.376883 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.377250 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.377256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.377259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.377510 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.377776 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.378060 (+ 284us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.378270 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":614,"threads_started":1}
I20260621 07:03:33.378700 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.366185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51924)
0621 07:03:33.366327 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.366342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.366455 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.366762 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.366766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.367203 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.367334 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.367342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.368319 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.368326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.369208 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.369219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.369901 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.369906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.370104 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.370652 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.370667 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.373058 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.376306 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.376312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.376316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.376654 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.376996 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.377002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.377005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.377126 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.377592 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.377596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.377734 (+ 138us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.377878 (+ 144us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.378318 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":146}
I20260621 07:03:33.378716 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.366067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.2.245.252:37101)
0621 07:03:33.366247 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.366257 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.366412 (+ 155us) server_negotiation.cc:407] Connection header received
0621 07:03:33.366587 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.366592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.366653 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.366740 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.367483 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.368186 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.369353 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.369742 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.369876 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.373227 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.373236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.373239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.373284 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.376166 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.376763 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.376771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.376774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.376893 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.377223 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.377230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.377233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.377481 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.377626 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.377947 (+ 321us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.378172 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:03:33.383467 3031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:03:33.384328 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: Bootstrap starting.
I20260621 07:03:33.387555 26013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:33.385825 25840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "19c513a0a8ec4d31b1b6a36830b0618b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57942
I20260621 07:03:33.388635 26013 log.cc:824] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:33.391732 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: No bootstrap required, opened a new log
I20260621 07:03:33.399121 26013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.399499 26013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627114969a894eea9d38909cfe72f025, State: Initialized, Role: FOLLOWER
I20260621 07:03:33.400054 26013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.400942 26021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.401274 26021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.402127 26013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: configured and running, proceeding with master startup.
W20260621 07:03:33.411875 26032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:03:33.412065 26032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:03:33.409552 25946 master_runner.cc:186] Error getting master registration for 127.2.245.254:43665: OK, instance_id { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" instance_seqno: 1782025412286879 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:03:33.417721 25946 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:03:33.432555 25865 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 4
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.254@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
I20260621 07:03:33.448180 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.433424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.2.245.252:37101)
0621 07:03:33.433624 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.433630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.433652 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:03:33.433833 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.433846 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.433916 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.434023 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.434800 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.435636 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.436868 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.437312 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.437452 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.441543 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.441549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.441554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.441597 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.445335 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.446176 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.446181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.446185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.446325 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.446758 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.446762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.446766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.447214 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.447361 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.447774 (+ 413us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.447935 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:03:33.448380 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.433319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51934)
0621 07:03:33.433542 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.433565 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.433697 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.434061 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.434065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.434497 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.434653 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.434661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.435767 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.435773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.436715 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.436727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.437466 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.437472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.437686 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.438344 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.438379 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.441330 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.445490 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.445496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.445500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.446053 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.446447 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.446452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.446456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.446638 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.447339 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.447344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.447527 (+ 183us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.447692 (+ 165us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.447998 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260621 07:03:33.453117 25865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: Bootstrap starting.
I20260621 07:03:33.457404 25865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:33.458659 25865 log.cc:824] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:33.461527 25865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: No bootstrap required, opened a new log
I20260621 07:03:33.467060 25865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.467438 25865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c513a0a8ec4d31b1b6a36830b0618b, State: Initialized, Role: FOLLOWER
I20260621 07:03:33.467935 25865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.468881 26035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.469298 26035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.470144 25865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:33.470242 25840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260621 07:03:33.470470 25840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:33.474085 25840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.477290 25840 leader_election.cc:288] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [CANDIDATE]: Term 1 election: Requested vote from peers 4584e68482494705a2e692e612046cba (127.2.245.253:41363), 627114969a894eea9d38909cfe72f025 (127.2.245.252:37101)
I20260621 07:03:33.478119 25988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "627114969a894eea9d38909cfe72f025"
I20260621 07:03:33.478627 25988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:33.483650 25988 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19c513a0a8ec4d31b1b6a36830b0618b in term 1.
I20260621 07:03:33.484280 25815 leader_election.cc:302] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19c513a0a8ec4d31b1b6a36830b0618b, 627114969a894eea9d38909cfe72f025; no voters:
I20260621 07:03:33.484797 26035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:03:33.491237 26035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [term 1 LEADER]: Becoming Leader. State: Replica: 19c513a0a8ec4d31b1b6a36830b0618b, State: Running, Role: LEADER
I20260621 07:03:33.491855 26035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.500727 26049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c513a0a8ec4d31b1b6a36830b0618b. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.501332 26049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:33.483649 25798 master_runner.cc:186] Error getting master registration for 127.2.245.253:41363: OK, instance_id { permanent_uuid: "4584e68482494705a2e692e612046cba" instance_seqno: 1782025412796064 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:03:33.504536 25798 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260621 07:03:33.509909 26046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:03:33.510092 26046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:03:33.511812 26051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:03:33.514066 26051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:03:33.521420 25939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } attempt: 5
Jun 21 07:03:33 dist-test-slave-9zdv krb5kdc[25785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782025412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.245.253@KRBTEST.COM for kudu/127.2.245.252@KRBTEST.COM
W20260621 07:03:33.545612 25967 tablet.cc:2369] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:03:33.531735 25819 tablet.cc:2369] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:03:33.564692 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.522097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.245.252:37101 (local address 127.0.0.1:51938)
0621 07:03:33.522560 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0621 07:03:33.522584 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0621 07:03:33.522712 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:03:33.523508 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:03:33.523516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:03:33.524212 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:03:33.524496 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.524512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.526639 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.526649 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.528731 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:03:33.528755 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.530528 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.530537 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:03:33.531117 (+ 580us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.532492 (+ 1375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:03:33.532549 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:03:33.549963 (+ 17414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:03:33.554776 (+ 4813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.554782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.554786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.558157 (+ 3371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.559576 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:03:33.559584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:03:33.559589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:03:33.562478 (+ 2889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:03:33.563203 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:03:33.563208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:03:33.563484 (+ 276us) client_negotiation.cc:769] Sending connection context
0621 07:03:33.563699 (+ 215us) client_negotiation.cc:240] Negotiation successful
0621 07:03:33.564146 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260621 07:03:33.566300 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.522637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.2.245.252:37101)
0621 07:03:33.522834 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.522875 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.522900 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:03:33.522999 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.523007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.523138 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.523267 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.524763 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.526374 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.529079 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.530012 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.530179 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.550531 (+ 20352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.550543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.550551 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.550596 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.554571 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.558606 (+ 4035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.558637 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.558643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.558976 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.562633 (+ 3657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.562641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.562646 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.563064 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.563209 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.565668 (+ 2459us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.565907 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":6528}
I20260621 07:03:33.569114 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.535400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.2.245.252:37101)
0621 07:03:33.540543 (+ 5143us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.540551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.549843 (+ 9292us) server_negotiation.cc:407] Connection header received
0621 07:03:33.550107 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.550113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.550206 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.550284 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.552828 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.554227 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.556221 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.556908 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.557029 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.560603 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.560610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.560615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.560652 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.564386 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.565588 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.565595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.565603 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.565784 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.566215 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.566220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.566224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.566847 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.567165 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.568486 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.568709 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4951,"thread_start_us":4757,"threads_started":1}
I20260621 07:03:33.573737 25988 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [term 1 FOLLOWER]: Refusing update from remote peer 19c513a0a8ec4d31b1b6a36830b0618b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:03:33.574824 26049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [LEADER]: Connected to new peer: Peer: permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:03:33.575766 25939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: Bootstrap starting.
I20260621 07:03:33.583634 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.531977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.2.245.254:43665)
0621 07:03:33.547487 (+ 15510us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.547496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.547525 (+ 29us) server_negotiation.cc:407] Connection header received
0621 07:03:33.547609 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.547617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.547719 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.547855 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.559064 (+ 11209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.560121 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.572219 (+ 12098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.572982 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.573082 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.576715 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.576722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.576728 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.576770 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.580302 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.581199 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.581205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.581208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.581399 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.581840 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.581846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.581849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.582196 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.582449 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.583027 (+ 578us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.583266 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260621 07:03:33.585575 25939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: Neither blocks nor log segments found. Creating new log.
I20260621 07:03:33.587162 25939 log.cc:824] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: Log is configured to *not* fsync() on all Append() calls
I20260621 07:03:33.588419 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.528188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.2.245.253:41363)
0621 07:03:33.528463 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.528471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.542761 (+ 14290us) server_negotiation.cc:407] Connection header received
0621 07:03:33.547132 (+ 4371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.547139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.547232 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.547309 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.549592 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.550662 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.555251 (+ 4589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.555950 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.556051 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.571396 (+ 15345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:03:33.571405 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:03:33.571412 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:03:33.571454 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:03:33.575074 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.581315 (+ 6241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.581320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.581323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.581481 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:03:33.582861 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:03:33.582869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:03:33.582876 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:03:33.583304 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:03:33.583589 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.587814 (+ 4225us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.588044 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:03:33.597245 25939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: No bootstrap required, opened a new log
I20260621 07:03:33.607479 26021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c513a0a8ec4d31b1b6a36830b0618b. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.607810 26021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.608997 25939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.609447 25939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4584e68482494705a2e692e612046cba, State: Initialized, Role: FOLLOWER
I20260621 07:03:33.609998 25939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } }
I20260621 07:03:33.621491 26021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.621841 26021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.625458 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.626339 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.626693 25939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:03:33.627401 25913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:03:33.628485 26049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.628749 26049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:33.628818 26051 catalog_manager.cc:1347] Generated new cluster ID: dea0f6cab3e84b7b9abfd324cd98bb24
I20260621 07:03:33.628916 26051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:03:33.631175 25913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [term 1 FOLLOWER]: Refusing update from remote peer 19c513a0a8ec4d31b1b6a36830b0618b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:03:33.632542 26035 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:03:33.626744 25914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4584e68482494705a2e692e612046cba"
I20260621 07:03:33.637023 25914 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19c513a0a8ec4d31b1b6a36830b0618b in term 1.
I20260621 07:03:33.662178 26051 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:03:33.665323 26051 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:03:33.689284 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c513a0a8ec4d31b1b6a36830b0618b. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.689682 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.690006 26070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.690506 26070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:03:33.690486 26049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c513a0a8ec4d31b1b6a36830b0618b" member_type: VOTER last_known_addr { host: "127.2.245.254" port: 43665 } } peers { permanent_uuid: "4584e68482494705a2e692e612046cba" member_type: VOTER last_known_addr { host: "127.2.245.253" port: 41363 } } peers { permanent_uuid: "627114969a894eea9d38909cfe72f025" member_type: VOTER last_known_addr { host: "127.2.245.252" port: 37101 } } }
I20260621 07:03:33.690852 26049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b [sys.catalog]: This master's current role is: LEADER
I20260621 07:03:33.696941 26051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c513a0a8ec4d31b1b6a36830b0618b: Generated new TSK 0
I20260621 07:03:33.698092 26051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:03:33.728606 26081 catalog_manager.cc:1259] Loaded cluster ID: dea0f6cab3e84b7b9abfd324cd98bb24
I20260621 07:03:33.728871 26081 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: loading cluster ID for follower catalog manager: success
I20260621 07:03:33.732908 26081 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: acquiring CA information for follower catalog manager: success
I20260621 07:03:33.735543 26081 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4584e68482494705a2e692e612046cba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260621 07:03:33.836818 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:33.823214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.2.245.252:37101)
0621 07:03:33.823458 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.823465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.834593 (+ 11128us) server_negotiation.cc:407] Connection header received
0621 07:03:33.835119 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.835126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.835239 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.835325 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.836555 (+ 1230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51960: BlockingRecv error: recv got EOF from 127.0.0.1:51960 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 07:03:33.839547 25869 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:33.829254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.2.245.254:43665)
0621 07:03:33.829443 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.829450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.837817 (+ 8367us) server_negotiation.cc:407] Connection header received
0621 07:03:33.838067 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.838072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.838181 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.838260 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.839258 (+ 998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57954: BlockingRecv error: recv got EOF from 127.0.0.1:57954 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260621 07:03:33.840910 26019 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:33.821594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.2.245.253:41363)
0621 07:03:33.821799 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.821805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.827443 (+ 5638us) server_negotiation.cc:407] Connection header received
0621 07:03:33.839216 (+ 11773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.839222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.839351 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.839421 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.840598 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32774: BlockingRecv error: recv got EOF from 127.0.0.1:32774 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:03:33.893254 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.876888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.2.245.253:41363)
0621 07:03:33.877092 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.877098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.884826 (+ 7728us) server_negotiation.cc:407] Connection header received
0621 07:03:33.885368 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.885375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.885610 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.885757 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:33.886507 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.887687 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.889977 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.890712 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.890816 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.891633 (+ 817us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:33.892385 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:33.892526 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.892781 (+ 255us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.892886 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260621 07:03:33.940246 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:33.927049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.2.245.252:37101)
0621 07:03:33.927249 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.927260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.938023 (+ 10763us) server_negotiation.cc:407] Connection header received
0621 07:03:33.938568 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.938576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.938716 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.938797 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:33.939967 (+ 1170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51970: BlockingRecv error: recv got EOF from 127.0.0.1:51970 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:03:33.998394 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:33.980494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.2.245.253:41363)
0621 07:03:33.980738 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:33.980744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:33.991583 (+ 10839us) server_negotiation.cc:407] Connection header received
0621 07:03:33.991790 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:33.991796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:33.991913 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:33.992025 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:33.992702 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.993848 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.995993 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:33.996765 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:33.996880 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:33.997536 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:33.997708 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:33.997769 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:33.997989 (+ 220us) server_negotiation.cc:299] Negotiation successful
0621 07:03:33.998075 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260621 07:03:34.029778 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.028153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.2.245.252:37101)
0621 07:03:34.028345 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.028353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.028373 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:03:34.028449 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.028454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.028551 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.028666 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.029542 (+ 876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51978: BlockingRecv error: recv got EOF from 127.0.0.1:51978 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:03:34.042739 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.035167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.2.245.253:41363)
0621 07:03:34.035403 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.035411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.035933 (+ 522us) server_negotiation.cc:407] Connection header received
0621 07:03:34.036232 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.036239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.036341 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.036444 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.036915 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.038165 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.040134 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.040836 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.040984 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.041834 (+ 850us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.042048 (+ 214us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.042144 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.042322 (+ 178us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.042421 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 07:03:34.059101 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.056682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.2.245.252:37101)
0621 07:03:34.056901 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.056907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.057452 (+ 545us) server_negotiation.cc:407] Connection header received
0621 07:03:34.057650 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.057656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.057772 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.057901 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.058824 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51986: BlockingRecv error: recv got EOF from 127.0.0.1:51986 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 07:03:34.074764 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.067618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.2.245.253:41363)
0621 07:03:34.067815 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.067826 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.068262 (+ 436us) server_negotiation.cc:407] Connection header received
0621 07:03:34.068469 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.068476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.068572 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.068688 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.069172 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.070375 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.072534 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.073249 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.073408 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.073875 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.074055 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.074148 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.074380 (+ 232us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.074469 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260621 07:03:34.089138 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.082164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.2.245.252:37101)
0621 07:03:34.082413 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.082421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.087574 (+ 5153us) server_negotiation.cc:407] Connection header received
0621 07:03:34.087726 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.087732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.087837 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.087967 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.088906 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51996: BlockingRecv error: recv got EOF from 127.0.0.1:51996 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
I20260621 07:03:34.107344 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.100392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.2.245.253:41363)
0621 07:03:34.100588 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.100596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.101141 (+ 545us) server_negotiation.cc:407] Connection header received
0621 07:03:34.101328 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.101334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.101453 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.101565 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.102011 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.103290 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.105198 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.105933 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.106060 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.106509 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.106639 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.106737 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.106906 (+ 169us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.107018 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260621 07:03:34.119561 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.117207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.2.245.252:37101)
0621 07:03:34.117393 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.117401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.117952 (+ 551us) server_negotiation.cc:407] Connection header received
0621 07:03:34.118151 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.118158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.118264 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.118395 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.119283 (+ 888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52012: BlockingRecv error: recv got EOF from 127.0.0.1:52012 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:03:34.147998 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.140931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.2.245.253:41363)
0621 07:03:34.141134 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.141140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.141724 (+ 584us) server_negotiation.cc:407] Connection header received
0621 07:03:34.141926 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.141932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.142033 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.142135 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.142620 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.143813 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.145830 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.146550 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.146676 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.147192 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.147334 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.147437 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.147608 (+ 171us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.147684 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 07:03:34.156997 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.154685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.2.245.252:37101)
0621 07:03:34.154903 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.154910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.155453 (+ 543us) server_negotiation.cc:407] Connection header received
0621 07:03:34.155606 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.155614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.155721 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.155843 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.156769 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52024: BlockingRecv error: recv got EOF from 127.0.0.1:52024 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:03:34.204149 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.196894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.2.245.253:41363)
0621 07:03:34.197085 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.197091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.197651 (+ 560us) server_negotiation.cc:407] Connection header received
0621 07:03:34.197980 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.197986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.198097 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.198210 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.198702 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.200005 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.201709 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.202461 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.202613 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.203204 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.203426 (+ 222us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.203524 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.203695 (+ 171us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.203784 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260621 07:03:34.212733 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.210583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.2.245.252:37101)
0621 07:03:34.210854 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.210860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.211249 (+ 389us) server_negotiation.cc:407] Connection header received
0621 07:03:34.211431 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.211435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.211503 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.211584 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.212494 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52026: BlockingRecv error: recv got EOF from 127.0.0.1:52026 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:03:34.286765 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.281241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.2.245.253:41363)
0621 07:03:34.281619 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.281628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.281933 (+ 305us) server_negotiation.cc:407] Connection header received
0621 07:03:34.282176 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.282180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.282249 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.282356 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.282790 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.283712 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.285054 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.285491 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.285634 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.286009 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.286127 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.286225 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.286380 (+ 155us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.286459 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
W20260621 07:03:34.294308 26059 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:03:34.292424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.2.245.252:37101)
0621 07:03:34.292613 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.292619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.292967 (+ 348us) server_negotiation.cc:407] Connection header received
0621 07:03:34.293099 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.293105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.293195 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.293309 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:03:34.294094 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52032: BlockingRecv error: recv got EOF from 127.0.0.1:52032 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 07:03:34.414149 26032 catalog_manager.cc:1259] Loaded cluster ID: dea0f6cab3e84b7b9abfd324cd98bb24
I20260621 07:03:34.414269 26032 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: loading cluster ID for follower catalog manager: success
I20260621 07:03:34.417052 26032 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: acquiring CA information for follower catalog manager: success
I20260621 07:03:34.419158 26032 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 627114969a894eea9d38909cfe72f025: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:03:34.431141 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.425670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.2.245.253:41363)
0621 07:03:34.425910 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.425918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.426274 (+ 356us) server_negotiation.cc:407] Connection header received
0621 07:03:34.426435 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.426439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.426509 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.426600 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.427075 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.427911 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.429420 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.429854 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.429956 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.430342 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.430455 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.430524 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.430720 (+ 196us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.430786 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 07:03:34.442271 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.436711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.2.245.252:37101)
0621 07:03:34.436954 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.436961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.437210 (+ 249us) server_negotiation.cc:407] Connection header received
0621 07:03:34.437432 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.437437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.437594 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.437713 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.438128 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.438819 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.440140 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.440530 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.440673 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.441136 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.441627 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.441800 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.441856 (+ 56us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.441935 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 07:03:34.466910 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.448222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.2.245.254:43665)
0621 07:03:34.459057 (+ 10835us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.459067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.460037 (+ 970us) server_negotiation.cc:407] Connection header received
0621 07:03:34.460287 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.460295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.460548 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.460744 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.461341 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.462463 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.464398 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.464832 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.464971 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.465532 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.465990 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.466100 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.466592 (+ 492us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.466677 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10634,"thread_start_us":10862,"threads_started":1}
I20260621 07:03:34.467693 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.448530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.2.245.252:37101)
0621 07:03:34.449213 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.449221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.460348 (+ 11127us) server_negotiation.cc:407] Connection header received
0621 07:03:34.460640 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.460647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.460767 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.460859 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.461643 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.462667 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.465164 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.465687 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.465779 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.466279 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.466423 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.466515 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.467322 (+ 807us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.467426 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543}
I20260621 07:03:34.467819 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:03:34.448700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.2.245.253:41363)
0621 07:03:34.448916 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0621 07:03:34.448925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:03:34.460616 (+ 11691us) server_negotiation.cc:407] Connection header received
0621 07:03:34.460902 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:03:34.460910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:03:34.461033 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:03:34.461105 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:03:34.461874 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.462830 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.465851 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:03:34.466400 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:03:34.466529 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:03:34.466978 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:03:34.467109 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:03:34.467187 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:03:34.467479 (+ 292us) server_negotiation.cc:299] Negotiation successful
0621 07:03:34.467565 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 07:03:34.472177 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25798
I20260621 07:03:34.473274 25860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:34.584369 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25798
I20260621 07:03:34.605486 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25872
I20260621 07:03:34.606680 25934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:34.713930 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25872
I20260621 07:03:34.734139 3031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu pid 25946
I20260621 07:03:34.735358 26008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260621 07:03:34.842017 3031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV0VCJ_/build/asan/bin/kudu with pid 25946
2026-06-21T07:03:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4414 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9315 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (357642 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST